[Mimedefang] Graphdefang + High CPU Load when processing

[Chris Gauch]

I ran into this same issue with Graphdefang, but it was fairly easy to
resolve.  I set up graphdefang on a remote Linux server that had a low
average load.

You can edit your /etc/syslog.conf file to send out all maillog info to a
remote syslog server by adding the following line to the syslog.conf file:
# Log all the mail messages in one place.
#mail.*      /var/log/maillog # <-- you can comment this line if you wish
mail.*       @ip.address.of.server # <-- add IP of your syslog server

On the remote Linux syslog server, install graphdefang, including all the
prerequisites, then edit your syslog init script (probably in
/etc/init.d/syslog) so that SYSLOGD_OPTIONS line looks like this:
SYSLOGD_OPTIONS="-m 0 -r"

The "-r" option allows remote syslog. You do not need to edit the
syslog.conf file on the MIMEDefang/mail server.

You can now run process your graphdefang stats on the remote server (since
the maillog info is now on the remote server), rather than your MIMEDefang
server (which already has enough to process). If your mail server is busy,
make sure your remote Linux server resides on your LAN, preferably a 100mbit
or faster network.  

You could also set up an rsync process to rsync the maillog onto the remote
graphdefang server, rather than configuring remote syslog.

On the remote syslog server running graphdefang, you should also add a CRON
script that runs the graphdefang.pl --trim option.  Just make sure this CRON
script DOES NOT run while graphdefang is processing the SummaryDB info, or
you'll corrupt the he|| out of your SummaryDB. 

- Chris 


> -----Original Message-----
> From: mimedefang-bounces at lists.roaringpenguin.com [mailto:mimedefang-
> bounces at lists.roaringpenguin.com] On Behalf Of Kevin A. McGrail
> Sent: Friday, August 06, 2004 10:42 AM
> To: mimedefang at lists.roaringpenguin.com
> Cc: jonas_lists at frukt.org
> Subject: Re: [Mimedefang] Graphdefang + High CPU Load when processing
> 
> I run it once a day and never have a corruption issue.  If you have
> corruption issues, suggest looking at your DB installation.  There is just
> very little in graphdefang that could really cause this issue.  Are you
> having other DB corruption issues?
> 
> I am very wary of using DB on servers because of the numerous issues we
> see
> but the speed benefits are great compared to something like mySQL.
> 
> In other words, all over different mailing lists, I constantly read about
> DB
> corrupted this, DB corrupted that.  Not to mention that DB is often
> implemented in a way that loads the entire DB into memory.  This is great
> for small databases but the graphdefang database can hit half a gigabyte
> for
> a server.  This causes huge spikes in the load but makes it process
> quicker.
> 
> Perhaps your machine is running out of memory trying to load the database
> and just crashing?
> 
> Regards,
> KAM
> 
> > I, too, have noticed that it is *very* easy to corrupt the graphdefang
> > database.
> 
> _______________________________________________
> Visit http://www.mimedefang.org and http://www.canit.ca
> MIMEDefang mailing list
> MIMEDefang at lists.roaringpenguin.com
> http://lists.roaringpenguin.com/mailman/listinfo/mimedefang