[Mimedefang] clamd memory leak?

[Kelsey Cummings]

On Fri, Apr 16, 2004 at 11:54:26PM -0500, Les Mikesell wrote:
> On Fri, 2004-04-16 at 22:24, Kelsey Cummings wrote:
> 
> > That's probably true but clamd has insane memory requirements for
> > processing deeply nested mime messages.  (See the thread about this for
> > more information.  David's hinted that he might patch the MIME module so
> > you can retrieve and nesting depth from it easily which could be used to
> > block excessively deep messages in MD.)
> 
> The interesting thing in my case was that a single run didn't
> seem to be so much of a problem memory-wise as that mimedefang
> timed out the connection.  However the memory wasn't completely
> released or re-used on subsequent runs so it kept growing as
> the sending server retried the tmp failures.

Clamd has definitely has had some pretty severe memory leaks.  Later CVS
snapshots are probably you best bet but .70rc has been okay for us.

BTW, it's not related to clamd but I thought I'd share it here.

We just hacked MD to query and report to a private DCC server mesh in order
to reduce spam influx.  (We were only using MD to wrap clamd and for some
of it's mime sanity features.)  The catch is that we've been doing spam
list poisioning for years and get around 250k spams a day to our trolling
accounts and that we feed these messages tagged as many to the servers
directly from MD.  If we see a message that is not sent to a troll and has
was sent to a troll we bounce it.  Were rejecting nearly 200k messages a
day based on this, many of the rejected spams are hard to filter url spams.

I hacked it in as a virus filter and pushed it in first so it runs before
our virus filters.  There some other small changes to get the virus code to
respect the 'spam' bounce.

sub message_contains_spam_localdcc () {

        my $is_troll = 0;
        my %virtusers;

        my $DB_HASH = new DB_File::HASHINFO;
        tie %virtusers, 'DB_File', '/etc/mail/virtusertable.db', O_RDONLY, 0666 , $DB_HASH;

        for my $recipient (@Recipients)
        {
                #strip '<...>'
                $recipient =~ s/^<//;
                $recipient =~ s/>$//;
                #lc it
                $recipient = lc $recipient;

                #md_syslog('err',"$MsgID: looking up $recipient in virtusertable");
                if ( exists $virtusers{$recipient} && $virtusers{$recipient} eq 'spamcatcher at xxx.net')
                {
                        $is_troll = 1;
                        next;
                }

                my ($lhs,$rhs) = split(/@/,$recipient);

                if ( exists $virtusers{"\@$rhs"} && $virtusers{"\@$rhs"} eq 'spamcatcher at xxx.net')
                {
                        $is_troll = 1;
                        next;
                }
        }
        untie %virtusers;

        my $cmd = '';
        if ( $is_troll )
        {
                $cmd = "/opt/dcc/bin/dccproc -ERt many -c all,0,-1 -l /opt/dcc/log -i ./Work/INPUTMSG";
                md_syslog('err',"$MsgID: Fed local dcc database.");
        }
        else
        {
                $cmd = "/opt/dcc/bin/dccproc -H -i ./Work/INPUTMSG";
        }

        unless( open DCC, "$cmd |" )
        {
                my $msg =  "Unable to execute $cmd: $!\n";
                md_syslog('err',"$MsgID: message_contains_spam_localdcc: $msg");
                return(return (wantarray ? (0, 'ok', 'ok') : 0));
        }

        my $results = <DCC>;
        close DCC;
        chomp($results);

        if ( $is_troll == 0 && $results =~ /=many/ )
        {
                md_syslog('err',"$MsgID: Bounce: $results");
                return ( wantarray ? (1, 'spam', 'reject') : 1);
        }

        return (wantarray ? (0, 'ok', 'ok') : 0);
}
 

-- 
Kelsey Cummings - kgc at sonic.net           sonic.net, inc.
System Administrator                      2260 Apollo Way
707.522.1000 (Voice)                      Santa Rosa, CA 95407
707.547.2199 (Fax)                        http://www.sonic.net/
Fingerprint = D5F9 667F 5D32 7347 0B79  8DB7 2B42 86B6 4E2C 3896