[Mimedefang] Graphdefang 0.9 missing events?

[John Kirkland]

>
> One question, though. What is this I hear about the SummaryDB eating near 1G
> of memory? :( That does not sound good. I expect to process/add ca. 5,000
> entries a day. Any idea how long the SummaryDB will last me, before it
> starts growing in the hundreds of megs?
>

WHAT?  I've never heard of this!  :-)  Just kidding...

Yes, for very large installations graphdefang can eat up a ton of memory.
I made some design decisions up front when I built graphdefang that were
fine for my use, but not so fine for some other people's use.

ASIDE:  The basic problem is that perl does not give memory back to the OS
until it exits -- even if you free it up in your program.  So, at specific
points in the program I have 2 copies of the SummaryDB in memory (to do db
backups & when saving the summarydb to disk).  And... the fix to this is
to change some of my design decisions.  I actually wanted to have the
whole hash in memory when I do my processing, but this has proven to be
problematic!  The problem I have with fixing this is simply TIME.  I leave
the country for work about once per month (mostly to Singapore & Armenia),
and I have not gotten to fixing this yet.

Back to your issue:  You shouldn't see any issues with 5,000 entries per
day.  I have graphdefang running very smoothly on sites with 10's of
thousands of messages per day.  Make sure you use the "--trim" option
fairly often.  It will cut out all old statistics that aren't relevant
anymore.

Regards,
John