[Mimedefang] really high cpu

[James Ebright]

Not to get into the basic Linux vs the world debate here (Otherwise known as 
my OS is better), but:

On Wed, 9 Feb 2005 02:55:31 -0800, John Nemeth wrote:

>>Nah, recent versions of Solaris can easily beat Linux. 

>>    Load is certainly important, but looking for other things such 
>>as whether the machine is swapping is more important.  If the machine 
>>swaps, get more RAM. 

>>    The ability to tune a server and eliminate bottlenecks is one thing 
>>that seperates professional system administrators from wannabes.  If you 
>>don't know what you're doing, then you can make any system run slow.  If 
>>you do know what you're doing, then you can make any system run fast as 
>>long as you have decent hardware and a decent OS. 


It depends, Have you benchmarked a recent Linux system on comparable 
hardware, like I said, you would be surprised. As far as using "server grade" 
hardware, that goes without saying. I managed a server farm of over 40 
servers, and 3 clusters, mixed OSs, primarily Solaris and Linux. My ISP had 
OVER 500,000 customers (yes, 1/2 million) and my region had over 56,000 email 
accounts in our cluster. We did WELL over 10,000 messages a day (granted, 
this was before MIMEDefang and spam was only becoming a major problem 
industry wide when I left). So, unless you have done some unbiased testing, 
you can not make the claim above, esp. when it is contrary to much anecdotal 
evidence on the net (which I also take with a grain of salt ;-) ).

MIMEDefang does seem to be RAM hungry but this goes without saying, if a 
server swaps out much at all, you need to add more RAM or look at your 
running porocesses to see what you need to cull. Once a system is tuned, the 
best metric to actively monitor (yes, trend them all, but actively monitor) 
is load, ALL your other metrics are related to load in some manner. Generally 
speaking, if your load is greater than the number of processors in your box, 
then you have processes waiting on CPU time (very generally, but this is 
probably the best way to look at it for some).

Also, if you are not running a DNS caching server on your sendmail box then 
you need to, DNS is a large bottleneck for sendmail, even saving the extra 
processes and memory to go out over a LAN to another DNS server and back is 
generally worse than running a local caching only daemon.

I mention the hosts file as it is a common mistake made by less experienced 
admins, eg:  one ISP we purchased had EVERY server they owned and every nodes 
IP listed in their hosts file that they updated from a cetralized database... 
the hosts file was close to 1MB whch is not all that large, but sendmail 
reads that file in something like 3 times for every single message that came 
in or out of that server... needless to say... it was causing speed and 
memory issues.

I am not saying I know all, and all of our testing was done with Solaris 8, 
which I referenced above, I have no experience with Solaris 9, the tuning 
sendmail book is a good reference as well, I did not think about it as I am 
more familiar with the more archaic bat book as that is what I had available 
at the time. We had Solaris 8 pretty fine tuned, we also had Linux fine tuned 
as well, and in our benchmarks and live tests Linux smoked Solaris as far as 
sendmail was concerned, it was not even close. 

At my current venture, we maintain 3 different mail servers including our 
own, the largest is still rather modest compared to the systems I used to 
maintain, but it comparable to the one you maintain: here are last weeks 
general stats for this box:

Date       Reject     Tx/Rx     Spam:     Virus 
31 Jan     61437      9093      339       112 
01 Feb     71665      9800      331       119 
02 Feb     61729      9643      376       122 
03 Feb     66060      9630      378       112 
04 Feb     67957      10453     490       125 
05 Feb     61849      5216      390       54 
06 Feb     63170      4819      411       46 
TOTAL      453867     58654 

Rejected mail either bounced from trusted RBLs (i.e. our own and a few 
others) or scored so high that we quaranteened and bounced them back (I do 
not believe "joejobs" are included in the above stats, which probably means 
user unknowns and 4xx rejects are also discluded but I could be wrong, those 
are typically a statistically insignificant portion of the totals though if 
they are or not and woudl belong in the reject column). 

The other categories are self explainatory, the spam one is mail tagged and 
delievered (scored high, but not so high as to outright bounce). As you can 
see, this server delivers 9-10k messages a day, but processes closer to 80k a 
day, it does not even break a sweat, I could double this mail load and still 
be fine with no adjustments to hardware and maybe a few software tweaks. The 
hardware you ask... a fairly modest Dell Poweredge server, dual 1G PIIIs and 
2.5 GB memory using the built in Perc raid card (which I don't care for). 
This box ran a load average last week of .31, yes, not even a one... oh, and 
it runs Linux.

I agree 100% though, if you don't know what you are doing, you can impede the 
performace of any system rather easily.



--
EsisNet.com Webmail Client