[Mimedefang] Off Topic - Mail Message and MySQL

[Phil Eschallier]

Benjamin;

I just posted asking David if his commercial product's filter directly
updates the database, or if it is some form of post processing.

I don't have hard numbers [yet], but we struggle to keep MD + SA running in
environments processing more than 1 milling e-mail per day.  I've written a
lot of perl / DBI code, and my [admittedly anecdotal] opinion is that
anything time consuming that can me removed from the MD filter would better
serve high volume sites.

You could try refactoring your current processing to move your DB updates to
a separate task.  Have the filter log to the file system, then have your
task run every 5 minutes from cron, loading data from the quarantine
directories then removing the file system entries.  This way your filter is
lighter weight ... and if your DB server is down, filtering will not slow or
stall and the cron job can just try again 5 minutes later.

I'm looking forward to David's observations as this has been implemented in
MD's sister product line.

... Phil

 
 

-----Original Message-----
From: mimedefang-admin at lists.roaringpenguin.com
[mailto:mimedefang-admin at lists.roaringpenguin.com] On Behalf Of schmaus
Sent: Wednesday, June 18, 2003 11:48 AM
To: mimedefang at lists.roaringpenguin.com
Subject: RE: [Mimedefang] Off Topic - Mail Message and MySQL


> I've been thinking about logging such messages to an RDBMS
> as well, though it's not been a high priority yet.  After
> some preliminary hacking and knowing what kind of load MD
> + SA put on high volume servers, I'm thinking that the
> better route is to use the quarantine functions with the
> filter akin to the samples in the distribution, and have a
> separate task (cron driven) that loads data from the
> quarantine directories.  I don't like the potential impact
> of DB connection overhead or down DB servers to the filter
> and mail throughput.  Just my $.02 ...
> ... Phil

Phil,

Your idea makes sense, but my mimedefang_filter already
inserts  all the other data (sender, recipient, score and
subject) to the DB.  The problem is when my web interface
goes to retrieve the data, including the raw message, I
would prefer to get it from the database and not the
MIMEDefang directory.  In other words, as messages are
quarantined, I would like to push them into a DB and not
have them saved in the MIMEDefang dir.

Benjamin
_______________________________________________
MIMEDefang mailing list
MIMEDefang at lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang