[Mimedefang] Graphdefang + High CPU Load when processing

[Rich West]

First off, graphdefang is an awesome tool.

I, too, have noticed that it is *very* easy to corrupt the graphdefang 
database.

As was mentioned by the author and a couple of others on the list, the 
trick to avoid it from being corrupted is to increase the frequency that 
you run graphdefang...

However, the cause of the database corruption is actually not the size 
of the database, the problem seems to come in when graphdefang has a lot 
to analyze in your maillog and then add to the database.  If there is 
too much to handle in the maillog, it craps out, and takes the database 
with it.

However, even running it every 1/2 hr on a medium load server, it causes 
the CPU utilization to spike up heavily.. and I mean heavily.

I would be interested to see if Jonas' version overcomes the high CPU 
load problem.. or if anyone else has seen the same problem.

-Rich

>On Tue, 27 Jul 2004 10:07:52 +0530, Ravi.P CMC Engr wrote:
>
>  
>
>>  Can you please provide me the steps in order to split the DB to
>>  avoid such problem in future.
>>    
>>
>
>Splitting the database will not avoid problems that comes from bugs or errors in the database or serializing modules. The reason  split the database is because it was a bit more efficient when using DWHDB as it gave me the opportuinty to store some stuff in a flat DB_File instead.
>
>  
>
>>How can i proceed?? give me a
>>  detailed procedure!!
>>    
>>
>
>You can check my version of Graphdefang at <http://whatever.frukt.org> if you want to see how I have replaced the seraializing with tying to DWHDB and split the database. I can hardly get more detailed than actually showing you my code.
>
>If you want to do it your own way, you could still get ideas from my code.
>
>If you don't want to use my code and you're not familiar enough with perl programming to do something similar yourself, I'd suggest that you do something simpler than what I did. Replacing the original serializing with YAML worked fine for me and is a pretty small and simple modification of the code. You can find the YAML module at CPAN.
>
>Regards
>/Jonas
>
>  
>