[Mimedefang] performance tuning once again

[tomvo at absi.be]

Hi,

a few questions about performance tuning for mimedefang, feel free to 
answer whatever question you know something more about than I do :)



if the output of "md-mx-ctrl histo " command on the busiest time of the 
day shows that let's half of the slaves have 0 messages scanned, does this 
mean that i can safely lower (MX_MAXIMUM parameter)

my box is a solaris machine with 2 GB of RAM, there's usually 1 GB free 
(as 'top' reports). (I average some 30K mails per day)
Can I still benefit from lowering the number of slaves in this case ?

I have implemented the following 'performance improving' features:
- /var/spool/MIMEDefang on a ram disk
- MX_EMBED_PERL=yes
- spamassassin only checks message of  <100K

I am also using the multiplexor queuing using these parameters:
# Multiplexor queue size -- default is 0 (no queueing)
MX_QUEUE_SIZE=100

# Multiplexor queue timeout -- default is 30 seconds
MX_QUEUE_TIMEOUT=240


How does this queuing actually work ? is it only used if all available 
slaves are in use ? or is  the queuing using available slaves ? I'm 
wondering if I have so many free slaves, can I benefit from this queuing ?

My main concern is that every know and then,i get these dreaded 'timeout 
before data read' errors (timeout of 10 minutes specified in sendmail.cf) 
So I assume this means that certain mails take longer to scan than 10 
minutes ?  But if i check my mimedefang stats, I see that most messages 
take hardly 5 seconds to scan:

16/10/2005:15:17:02 1129468622.815 StartFilter slave=4 nslaves=20 nbusy=2
16/10/2005:15:17:03 1129468623.548 EndFilter slave=4 nslaves=20 nbusy=1 
numRequests=42
16/10/2005:15:17:05 1129468625.654 StartFilter slave=4 nslaves=20 nbusy=2
16/10/2005:15:17:09 1129468629.441 EndFilter slave=4 nslaves=20 nbusy=1 
numRequests=43


(I assume here that the above output means that mimedefang took 1 second 
to scan the first mail 15:17:02 until 15:17:03, and the second message 4 
seconds)
If my machine was really overloaded, shouldn't i be seeing much higher 
averages of seconds / mail scanned ? And does the above timings take into 
account the complete path from accepting the mail from sendmail to 
mimedefang, running clamav, and other checks enabled in mimedefang, 
passing the mail to spamassassin and let SA do it's various checks, and 
finally have mimedefang passing the mail back to sendmail ?


Also, about  this parameter:

# Number of seconds a process is allowed to scan an email before it is
# considered dead.  The default is 30 seconds; we suggest 600.
MX_BUSY=600


What happens if this limit is exceeded ? do i also get a 'please try again 
later' error ? I would like to configure my mail relay in such a way that, 
when for some reason, the mail couldn't be scanned in time, the mail is 
accepted anyway (because we have user's dropping mail directly to the mail 
relay from their mail client's, and if they receive such a 'please try 
again later' error, they get annoyed.


Thanks for your replies. Documentation on mimedefang and spamassassin is 
sometimes so overwhelming that I can't seem to understand the basics i'm 
afraid.

thx,
tom.



---------------------------------------------------------------------------
Tom Van Overbeke - ABSI Unix System Engineer
email: tomvo at absi.be
Tel: +32 2 333 40 00 - Fax: +32 2 332 34 69
website: http://www.absi.be
---------------------------------------------------------------------------