[Mimedefang] Comments on mimedefang.plL

[Clayton, Nik [IT]]

> >From an efficiency point of view, the code in serverloop() 
> > that parses the SENDER, HOSTIP, HOSTNAME, RECIPIENTS, and 
> > other files isn't always necessary.

[...]

> That's a good idea, but I doubt it has much of a performance 
> impact compared to calling SpamAssassin.  

My main concern is the additional disk accesses that are required.

> > Is there any interest in allowing filter_begin() to abort processing
> > the rest of the message?  If I've used SpamAssassin to determine
> > that the message is spam in filter_begin(), I don't want filter() to
> > be called for the rest of the parts in the message.
> 
> You can put this code right at the top of filter():
> 
> 	# Don't do work for nothing
> 	return if (message_rejected());

That's not the overhead I'm concerned about.  If I've rejected the
message in filter_begin(), there's no need for MimeDefang to go to all
the effort of parsing out the various MIME entities -- that can be
skipped entirely.

> > Now for a more general muse -- how do people feel about the 
> > 'interface' thatthe filter*() procedures have to the rest of the code?
> 
> It's bad. :-)  MIMEDefang accreted features; it's become 
> pretty messy now.
> I agree with you (and others have suggested) that a proper Perlish OO
> interface is needed.  If anyone wants to pay me to do it... :-)

Perhaps a mimedefang-dev mailing list, for API discussion?  Or would it
be on-topic here?

> > How about putting the user supplied filter subs in their 
> > own package, sothey've got their own namespace to play with?
> 
> You can do that in your filter.  Only filter, filter_begin, 
> filter_end, filter_multipart and defang_warning need to be in 'main'; 
> you could use your own package for everything else.

That's my current kludge :-)

> Questions for you:  Are you able to release any more details of how
> you handle such a high volume of e-mail?  Number and configuration of
> machines, etc?

Some 6 CPU E4500's run MimeDefang and SpamAssassin, talking to Sendmail 
over a TCP Milter connection.  I run mimedefang-multiplexor with 
"-m 10 -x 150" which is enough to handle our occasional spikes of mail.

The biggest performance improvement was simply to make 
/var/spool/MIMEDefang a symlink to /tmp/MIMEDefang -- /tmp is memory
backed.  Before doing this, vmstat/iostat showed that the limiting
factor was IO throughput -- the system couldn't keep up with the huge
number of small, temporary files that were being created and removed
with each message, and I'd regularly have 30 or 40 processes blocking
on IO.

Now that's not an issue.

> Also, are the rate-limiting switches in the multiplexor (-w 
> and -W) useful under heavy loads?

Haven't needed to use them.

N
-- 
1234567890123456789012345678901234567890123456789012345678901234567890123456
7890
  -- The 80 column-ometer

Global Messaging      Victoria Plaza, 2nd Floor     x21206