[Mimedefang] really high cpu

John Nemeth jnemeth at victoria.tc.ca
Wed Feb 9 05:55:31 EST 2005


On Jun 24, 10:52am, "James Ebright" wrote:
} On Mon, 31 Jan 2005 14:08:17 -0800 (PST), Stephen Ford wrote
} > mimedefang is using ~45% of my cpu which has me scared

     Is there anything else running on the box?  If not, then this
isn't a good measure since the OS will give all the CPU to the only
task.

} > since there isn't any mail running through the box
} > except mine and I'm suppose to use this box for the
} > campus staff of over 1000 employee's in a few weeks. 
} > I'm not getting much of a delay when I send mail that
} > isn't spam (less then a second) and spam is delayed
} > for ~10 seconds.
} > 
} > I'm running Solaris 9 with 2 gigs of ram.  I have
} > /var/spool/MIMEDefang in a 1 gig swap partition.  I

     How many MIMEDefang slaves are you using?  Keep in mind that you
should allocate at least 20 MB of RAM for each slave.  Again, is there
anything else running on the box that is consuming memory?  Is a
graphical interface running?  If so, get rid of it, this is a server
and it is needlessly using memory and CPU time..  Is the machine
swapping?  If so, can you decrease the number of things running so that
it uses less RAM.  If not, get more RAM.

} > installed the newest mimedefang this afternoon and the
} > newest spamassassin last Friday through CPAN.  I took
} > out the blacklist and uriblacklist which brought the

     Sounds like you have a DNS problem.  Make sure you have a local
DNS server that responds quickly.

} > box to it's knee's but it wasn't doing that when I was
} > running spamassassin 3.0.
} > 
} > Any suggestions before I throw this on a bigger box?
}
} Solaris just is not the best platform for running sendmail on, I have 

     Oh, nonsense!

} experience with E-450 boxes (as well as ultra-sparcs) running Solaris 8 (top 
} end solaris at the time) and honestly, I could bring that beefy box to its 
} knees just by running billing and letting the little pentium 300 pump out the 
} 20,000 or so messages one after another... Linux on the other hand on a 
} modest dual processor 500 MGhz, 512MB ram hardly broke a sweat. (typical pc 

     When it comes to multiprocessing Solaris can run circles around
Linux and most other OSes.

     BTW, I wouldn't even think of doing mission critical mail on a
typical PC.  They simply aren't server class hardware, i.e. they have
non-parity memory and often have cheap and/or low quality components.
If you're going to use a PC, make sure it is server class hardware
(i.e. high quality components, ECC RAM, etc.).

} The Solaris box constantly ran a load of 4-6 (4 processors, so this wasn't 
} too terrible) but spam runs could shoot the load up over 30.

     Prior to using MIMEDefang, I had mail services (including a bunch
of virtual domains) on a 143 MHz Ultra 1 with 224 MB of RAM.  It had a
random assortment of hard drives purchased over time.  It handled on
the order of 4000 accounts.  It also had login services, MySQL for
accounting, RADIUS, Apache with a bunch of virtual servers, FTP for web
site maintenance, POP3, NIS+, and several NFS exports.  Spam detection
was handled through procmail and spamassassin.  It processed about
10,000 messages per day without breaking a sweat.  It was not a beefy
machine by any stretch of the imagination.

     Now that I have MIMEDefang, I moved to a bigger server, an E4500
with eight 336 MHz processors and 4 GB of RAM, with the same random
assortment of hard drives (the next hardware upgrade I want to do is to
upgrade to a RAID array).  It runs all of the above plus ClamAV and
File::Scan for antivirus.  MIMEDefang's stream_by_recipient() function
is used to allow for individual spam preferences which are stored in a
MySQL database.  Because of stream_by_recipient() a significant number
of pieces of mail get processed multiple times.  It processes 25,000+
messages per day without breaking a sweat (load is usually a litte
under 1.0 and rarely spikes).  Although, it has a lot of processors,
each processor isn't very fast.  Your E450 could probably run circles
around it.

} The Linux box ran with a load of 1-2 (2 processors so again.. not terrible, 
} but better than Solaris) and spam spikes shot the load up to 4-8...
} 
} The difference I think is the OS and not hardware as we ran a few ad-hoc 

     The OS is fine if you know what you are doing.

} tests using the intel version of Solaris and found similar yet more 
} pronounced results. We never did try Linux on the E-450s as they were all 
} production boxes.
} 
} So, if you are stuck with Solaris... get out the bat book and optimize 

     There is a book out that is specifically about tuning sendmail,
"sendmail Performance Tuning" by Nick Christenson published by Pearson
Education with ISBN 0321115708.

} sendmail as much as you can, local DNS caching deamon helps alot as does a 

     My DNS server is on the same segment.  I don't have a local
caching daemon.  It is running on a Sparc IPC (40 MHz Sparc processor,
not even UltraSparc) with 32 MB of RAM, along with several other services.

} small as possible local host file. Try and avoid gigantic passwd files too, 

     This goes without saying.  You really should be using DNS and not
hard coding stuff.  The latter makes network maintenance and
troubleshooting a lot harder.

} NIS+ can help there. Turn off NFS and NFS automounters if possible. Tune the 

     This also goes without saying.  Having a large text file with
which you will constantly be doing linear searches is rather dumb.
That is pretty much guaranteed to bog down a system.  Solaris' nscd can
help, but can't really offset a multi-thousand line passwd file.  I
have several NFS exports and the standard NFS automounter for looping
back /export/home.

} disk IO as much as possible, if I remember right Solaris was optimized for 
} very large files, which mail is typically not. Play with tyhe nice level for 

     That's changing.

} mimedefang.  And if that all fails, yeah, you may need to up your hardware.

     With something like MIMEDefang it is important to have a lot of
memory.

} If you are not stuck with Solaris.... you might want to benchmark a modest 
} linux box.. you may be surprised.. esp with the 2.6 kernel.

     Nah, recent versions of Solaris can easily beat Linux.

} Also, load is a MUCH better indicator than %used on a processor, especially 
} in multi-processor systems as you have to factor in nice levels, system idle% 
} user levels, and several other factors that load factors in. If this box is 
} ONLY used for mail then load should be your primary metric.

     Load is certainly important, but looking for other things such
as whether the machine is swapping is more important.  If the machine
swaps, get more RAM.

     The ability to tune a server and eliminate bottlenecks is one thing
that seperates professional system administrators from wannabes.  If you
don't know what you're doing, then you can make any system run slow.  If
you do know what you're doing, then you can make any system run fast as
long as you have decent hardware and a decent OS.

}-- End of excerpt from "James Ebright"



More information about the MIMEDefang mailing list