[Mimedefang] performance scaling problems

[Chris Myers]

----- Original Message ----- 
From: "Lucas Albers" <admin at cs.montana.edu>
To: <mimedefang at lists.roaringpenguin.com>
Sent: Friday, January 30, 2004 4:31 PM
Subject: [Mimedefang] performance scaling problems


> Items I have seen for speedingup performance are:
> multiple queues, multiple queue runners, and configuring aggressive
> timesouts for sendmail.mc.
> Run fastest virus scanners first. Run daemonized versions of virus
scanners.
> User ext3 or reiserfs as the filesystem.
> Turn off acess time updates to the mail spool. Set noatime.
>
> Any items I am overlooking for Mimedefang/Sendmail scaling?

Put /tmp, /etc/mail/spamassassin/bayes and /var/spool/MIMEDefang in
memory-based filesystems.  Do NOT put /var/spool/mqueue in a memory
filesystem.  Back up the bayes database every day or so, and restore the
backup whenever the system reboots.

The SpamAssassin Bayesian database is the source of a LOT of I/O.

Fingerprint major mass-mailing viruses (subject, to/from, size, attachment
name, whatever) and check for the fingerprint BEFORE virus-scanning.  Drop
these messages.

IN PRAISE OF DAEMONIZED VIRUS-SCANNERS.  The command-line virus scanners are
multiple orders of magnitude slower than daemonized versions.  Avoid the
command-line scanners like the plague.

<BEGIN DISGUSTING HAM-HANDED BRUTE-FORCE SHELL CODE>

#! /bin/sh

cd /var/spool/mqueue

egrep -l '^S(<>|MAILER-DAEMON)' qf* | xargs egrep -l '^N[0-9]{2,}' | sed
's/qf/rm -f ??/; s/:.*//'

egrep -l '^M.*(does not resolve|Domain not found)' qf* | xargs egrep -l
'^N[0-9]{2,}' | sed 's/qf/rm -f ??/; s/:.*//' | sh

find /var/spool/mqueue -type f -mtime +5 -print | xargs rm

</BEGIN DISGUSTING HAM-HANDED BRUTE-FORCE SHELL CODE>

1. This script finds bounces where sendmail has attempted deliver 10 or more
times and deletes the files from the queue.
2. This script more aggressively looks for unresolvable e-mail addresses and
deletes the files from the queue.
3. This script finds anything older than 5 days in the queue and deletes the
files from the queue.
4. Yes, I know this is a terrible way to do it.  It's also effective.
5. Beware line wrapping, each "egrep ..." statement is a single line of
code.

Chris Myers
Networks By Design