[Mimedefang] mount noatime (was: ramdisks on Linux)

[Chris Myers]

----- Original Message ----- 
From: "Kenneth Porter" <shiva at sewingwitch.com>
To: <mimedefang at lists.roaringpenguin.com>
Sent: Wednesday, March 17, 2004 3:09 AM
Subject: [Mimedefang] mount noatime (was: ramdisks on Linux)


> --On Tuesday, March 16, 2004 5:09 PM -0600 Chris Myers
> <chris at by-design.net> wrote:
>
> > Mount your disks with the "noatime" option to cut down on largely
useless
> > disk writes.
>
> Good idea on a highly-loaded mail-only server.
>
> This does have the drawback that you can't easily see when your spools
have
> been popped, and you have to resort to parsing your POP server's logs for
> that. I've been using "ls -lut" to see which users don't collect their
> mail, usually a sign of someone leaving the company and HR forgetting to
> tell me about it.

Good point.

So, summarizing some of the previous messages...  If you looking for all
reasonable ways to maximize system performance:

Mount / as noatime
Mount /etc/mail as ramdisk, noatime [restore backup at boot time]
Mount /tmp as ramdisk, noatime
Mount /var as noatime
Mount /var/spool/MIMEDefang as ramdisk, noatime.
Mount /var/spool/mqueue/qf as noatime [on disk 1]
Mount /var/spool/mqueue/df as noatime [on disk 2]
Mount /var/spool/mqueue/xf as ramdisk, noatime [only if you split the
qdir's]
Mount /var/mail normally

Remember to make periodic backups of /etc/mail so you have something to
restore after a crash or reboot.

SCSI disks are notably faster than IDE disks.  Striping is notably faster
than not striping.  Higher RPM disks are faster than lower RPM disks (but 2x
the RPM does not mean 2x the actual performance).  Hardware RAID is faster
than software RAID.  IDE RAID is a low-cost option, and there are 10K RPM
IDE disks now.  IDE is fine until you're trying to push >>100K messages/day
or really big surges.

Between MIMEDefang and all of these ramdisks, you'll need a lot of memory.
At least 2GB of RAM.  If you start paging and swapping, performance will
drop considerably (keep in mind that disk is something like 1000x slower
than RAM!).

Set MX_REQUESTS to something like 500 and use embedded Perl if it works
under your O/S version.

As seen in the recommended filter, don't run SpamAssassin on messages larger
than 100K.

Use greylisting.  If your e-mail correspondents don't use Novell Groupwise,
Communigate Pro or Symantec Raptor firewalls you can use the MUCH more
efficient "tempfail after RCPT TO" form of greylisting.  Otherwise take note
that the message is greylisted and then tempfail the message at the top of
filter_begin.  IMPORTANT NOTE: action_tempfail just makes a note to tempfail
the message, it doesn't happen right then so you _should_ do "return if
message_rejected();" at the top of filter, filter_multipart and filter_end.

Drop executable attachments, don't bother to virus-scan them ... just drop
them.

Use daemonized virus scanners.  I've seen clamd scan a message in about 10%
of the CPU time it took clamscan to scan a message.  Don't bother to scan a
message you know will be rejected.

Validate all input (HELO, MAIL FROM, RCPT TO) as much as possible.  Read all
the various threads over the last couple months that cover what constitutes
"too much validation".  These tests may let you reject 10-15% of incoming
mail as spam without ever receiving the message body (a _big_ win).

Run a local caching nameserver on your filter.  That will get rid of a lot
of network traffic to the DNSBL's and generally improve performance a bit.

Use dccifd instead of dccproc, you'll save on an exec() for every scanned
message.  Consider running a DCC server locally, the public servers are
always overloaded.

Consider dropping messages that exceed a predetermined SpamAssassin score
(10-20 is a good range, it depends on your environment).  If you don't
deliver the message, that's just one less CPU/memory/disk hit.

CPU is important, but not as important as you might think.  A single P4
2.0GHz can handle more than 100K messages/day if the rest of the system is
balanced.  Don't go for that quad-processor 3.2GHz Xeon system with 4MB
on-die cache with 8 SCSI disks in a RAID 0+1 array, 32GB RAM and dual
gig-Ethernet NICs unless you're trying to figure out how to push a million
messages/day through one box.  Frankly it'll be cheaper to have a dozen 1u
P4 2.8GHz 40GB IDE systems than it would be to buy that one monster box ...
AND you'll have better reliability with a dozen expendable boxes.
Greylisting becomes slightly more interesting over a dozen boxes, you better
know (or learn) to use a real database system at that point.

Should we make "how to make a system running MIMEDefang go faster?" a FAQ
entry?

Chris Myers
Networks By Design