[Mimedefang] Graphdefang + High CPU Load when processing
Rich West
Rich.West at wesmo.com
Tue Aug 10 12:34:00 EDT 2004
Ya know, I was considering the very same thing. My backup server does
basically nothing throughout the day (it only runs backups), which makes
it a great candidate. :)
Just curious, but what, exactly, does the --trim option do? The
documentation states that it cuts out old data from the SummaryDB, but
doesn't that defeat the purpose of graphdefang when looking for
longer-term trends and such?
With regards to the corruption issue, the problem creeps in when
Graphdefang has a LOT of information to process. In my situation, we
have a medium load mail server which generates maillogs on a daily basis
anywhere between 2MB and 10MB. Now, that doesn't seem to be too large,
but, however, GraphDefang runs, when I was running it on a daily basis,
after about a month of gathering data (maybe sooner, but it's been
nearly 10 months since we were running it on a daily vs. every 30 minute
basis), we noticed that the graphs suddenly flatlined.
A little investigation showed that the database was corrupted (ran
graphdefang.pl by hand, and no new data would get added to the
database). That was the 4th occurance of the same problem, so we opted
for the more frequent updates to reduce the amount of data it had to
handle at any one point in time. This resolved the problem, but, now,
10 months later, we have seen that graphdefang has started spiking the
CPU for a minute or more as it processes (this is a new problem for
us). The machine is a solid box with 1GB of RAM..
I'll try running it on a remote server..
:-)
-Rich
Chris Gauch wrote:
>I ran into this same issue with Graphdefang, but it was fairly easy to
>resolve. I set up graphdefang on a remote Linux server that had a low
>average load.
><...snip...>
>You could also set up an rsync process to rsync the maillog onto the remote
>graphdefang server, rather than configuring remote syslog.
>
>On the remote syslog server running graphdefang, you should also add a CRON
>script that runs the graphdefang.pl --trim option. Just make sure this CRON
>script DOES NOT run while graphdefang is processing the SummaryDB info, or
>you'll corrupt the he|| out of your SummaryDB.
>
>- Chris
>
>
>
> Kevin A. McGrail wrote:
>
>I run it once a day and never have a corruption issue. If you have
>corruption issues, suggest looking at your DB installation. There is just
>very little in graphdefang that could really cause this issue. Are you
>having other DB corruption issues?
>
>I am very wary of using DB on servers because of the numerous issues we see
>but the speed benefits are great compared to something like mySQL.
>
>In other words, all over different mailing lists, I constantly read about DB
>corrupted this, DB corrupted that. Not to mention that DB is often
>implemented in a way that loads the entire DB into memory. This is great
>for small databases but the graphdefang database can hit half a gigabyte for
>a server. This causes huge spikes in the load but makes it process quicker.
>
>Perhaps your machine is running out of memory trying to load the database
>and just crashing?
>
>Regards,
>KAM
>
More information about the MIMEDefang
mailing list