[Mimedefang] mimedefang 2.19 crashed

Wed Aug 28 15:34:00 EDT 2002

Yesterday, I upgraded from mimedefang 2.16 to 2.19.  At 7am this morning
mimdefang crashed with the following error in maillog:

Aug 28 06:59:22 chaplin mimedefang[444]: Failure in cleanup line 1236: Not
enough space

After that sendmail started sending tempfails (I've since changed sendmail
to not do this). Looking back in the log I noticed a lot of:

Aug 28 00:01:45 chaplin mimedefang-multiplexor: Slave died prematurely --
check your filter rules
Aug 28 00:01:45 chaplin mimedefang[444]: Error from multiplexor: ERR No
response from slave
Aug 28 00:01:45 chaplin mimedefang-multiplexor: Reap: Idle slave 18 (pid
25973) exited due to signal 11 (SLAVE DIED UNEXPECTEDLY)

I ran mimedefang.pl -f /etc/mail/mimedefang-filter test and it didn't seem
to have a problem with it.  I didn't seen any difference between the
suggested filters in the two versions, so I did not change anything in my
filter (included below).  I did make one change to mimedefang.pl at line
1011.  I commented out the send_mail call because I don't need to get an
e-mail when something is quarantined (>1000 a day).  I have moved back to
2.16 without any errors showing up.

This a dual-processor Sun box running Solaris 7.  I'm using the multiplexor
with a minimum of 10 and max of 40 slaves.  Fortunately, last week I
configured Big Brother to start collecting system resource stats using
rrdtool.  At the same time that mimedefang died there is a small spike in
CPU (load ave 0.3 to 1.6) which might suggest a large file being parsed but
the files left over in /var/spool/mimedefang are not large.  Also, there was
no shortage of disk or ram space at anytime.  There was a small spike in
number of swap pages read, but there were larger spikes the previous day
without any problems.

Any ideas?

Kevin O'Brien
Systems/Network Administrator
E! Networks, Inc.
323.692.4742
323.692.4700 fax

# -*- Perl -*-
# $Id: mimedefang-filter,v 1.11 2002/08/21 17:23:41 kevino Exp $
# $Source: /code/cvsrep/etc/mimedefang/mimedefang-filter,v $
#***********************************************************************
#
# mimedefang-filter
#
# Suggested minimum-protection filter for Microsoft Windows clients, plus
# SpamAssassin checks if SpamAssassin is installed.
#
# Copyright (C) 2002 Roaring Penguin Software Inc.
#
# This program may be distributed under the terms of the GNU General
# Public License, Version 2, or (at your option) any later version.
#
#***********************************************************************

#***********************************************************************
# Set administrator's e-mail address here.  The administrator receives
# quarantine messages and is listed as the contact for site-wide
# MIMEDefang policy.  A good example would be 'defang-admin at mydomain.com'
#***********************************************************************
$AdminAddress = 'defang-admin at eonline.com';
$AdminName = "MIMEDefang Administrator";

#***********************************************************************
# Set the e-mail address from which MIMEDefang quarantine warnings and
# user notifications appear to come.  A good example would be
# 'mimedefang at mydomain.com'.  Make sure to have an alias for this
# address if you want replies to it to work.
#***********************************************************************
$DaemonAddress = 'mimedefang at eonline.com';

#***********************************************************************
# If you set $AddWarningsInline to 1, then MIMEDefang tries *very* hard
# to add warnings directly in the message body (text or html) rather
# than adding a separate "WARNING.TXT" MIME part.  If the message
# has no text or html part, then a separate MIME part is still used.
#***********************************************************************
$AddWarningsInline = 0;

#***********************************************************************
# Set various stupid things your mail client does below.
#***********************************************************************

# Set the next one if your mail client cannot handle nested multipart
# messages.  DO NOT set this lightly; it will cause action_add_part to
# work rather strangely.  Leave it at zero, even for MS Outlook, unless
# you have serious problems.
$Stupidity{"flatten"} = 0;

# Set the next one if your mail client cannot handle multiple "inline"
# parts.
$Stupidity{"NoMultipleInlines"} = 0;

# Size of message to skip spam checks
$SizeSkip = '100';

# Local networks to skip check when they are the relaying host
$LocalNetworks = '10\.|12\.46\.7|216\.136\.194|63\.240\.204';

# Text to add to a message with a quarantined attachment
$GeneralWarning = "";

# This procedure returns true for entities with bad filenames.
sub filter_bad_filename ($) {
  my($entity) = @_;
  my($bad_exts, $re);

  # Bad extensions
  $bad_exts =
'(ade|adp|bas|bat|chm|cmd|com|cpl|crt|dll|exe|hlp|hta|inf|ini|ins|isp|jse?|l
ib|lnk|mdb|mde|msc|msi|msp|mst|ocx|pcd|pif|reg|scr|sct|shb|shs|sys|url|vb|vb
e|vbs|vxd|wsc|wsf|wsh)';

  # Do not allow:
  # - curlies
  # - bad extensions (possibly with trailing dots) at end or
  #   followed by non-alphanum
  $re = '(\{)|(\})|(\.' . $bad_exts . ')\.*([^-A-Za-z0-9_.]|$)';
  return re_match($entity, $re);
}

# Scan for a virus using the first supported virus scanner we find.
sub message_contains_virus () {
    return message_contains_virus_avp()      if ($Features{'Virus:AVP'});
    return message_contains_virus_fprot()    if ($Features{'Virus:FPROT'});
    return message_contains_virus_fsav()     if ($Features{'Virus:FSAV'});
    return message_contains_virus_hbedv()    if ($Features{'Virus:HBEDV'});
    return message_contains_virus_nai()      if ($Features{'Virus:NAI'});
    return message_contains_virus_rav()      if ($Features{'Virus:RAV'});
    return message_contains_virus_sophie()   if ($Features{'Virus:SOPHIE'});
    return message_contains_virus_sophos()   if ($Features{'Virus:SOPHOS'});
    return message_contains_virus_trend()    if ($Features{'Virus:TREND'});
    return message_contains_virus_filescan() if
($Features{'Virus:FileScan'});
    return (wantarray ? (0, 'ok', 'ok') : 0);
}

# Scan for a virus using the first supported virus scanner we find.
sub entity_contains_virus ($) {
    my($e) = @_;
    return entity_contains_virus_avp($e)      if ($Features{'Virus:AVP'});
    return entity_contains_virus_fprot($e)    if ($Features{'Virus:FPROT'});
    return entity_contains_virus_fsav($e)     if ($Features{'Virus:FSAV'});
    return entity_contains_virus_hbedv($e)    if ($Features{'Virus:HBEDV'});
    return entity_contains_virus_nai($e)      if ($Features{'Virus:NAI'});
    return entity_contains_virus_rav($e)      if ($Features{'Virus:RAV'});
    return entity_contains_virus_sophie($e)   if
($Features{'Virus:SOPHIE'});
    return entity_contains_virus_sophos($e)   if
($Features{'Virus:SOPHOS'});
    return entity_contains_virus_trend($e)    if ($Features{'Virus:TREND'});
    return entity_contains_virus_filescan($e) if
($Features{'Virus:FileScan'});
    return (wantarray ? (0, 'ok', 'ok') : 0);
}

#***********************************************************************
# %PROCEDURE: filter_begin
# %ARGUMENTS:
#  None
# %RETURNS:
#  Nothing
# %DESCRIPTION:
#  Called just before e-mail parts are processed
#***********************************************************************
sub filter_begin () {
  $FromInternal = ($RelayAddr =~ /^(127.0.0.1|$LocalNetworks)/);

  if ($FromInternal) {
    action_add_header("X-MIMEDefang-Status", "Skipped, internal relay:
$RelayAddr");
    return action_accept();
  }

  # ALWAYS drop messages with suspicious chars in headers or body
  if ($SuspiciousCharsInHeaders) {
    action_quarantine_entire_message();
    #action_notify_administrator("Message quarantined because of suspicious
characters in headers");
    # Do NOT allow message to reach recipient(s)
    return action_discard();
  }

    # Scan for viruses if any virus-scanners are installed
    my($code, $category, $action) = message_contains_virus();
    $FoundVirus = ($category eq "virus");

}

#***********************************************************************
# %PROCEDURE: filter
# %ARGUMENTS:
#  entity -- a Mime::Entity object (see MIME-tools documentation for
details)
#  fname -- the suggested filename, taken from the MIME Content-Disposition:
#           header.  If no filename was suggested, then fname is ""
#  ext -- the file extension (everything from the last period in the name
#         to the end of the name, including the period.)
#  type -- the MIME type, taken from the Content-Type: header.
#
#  NOTE: There are two likely and one unlikely place for a filename to
#  appear in a MIME message:  In Content-Disposition: filename, in
#  Content-Type: name, and in Content-Description.  If you are paranoid,
#  you will use the re_match and re_match_ext functions, which return true
#  if ANY of these possibilities match.  re_match checks the whole name;
#  re_match_ext checks the extension.  See the sample filter below for
usage.
# %RETURNS:
#  Nothing
# %DESCRIPTION:
#  This function is called once for each part of a MIME message.
#  There are many action_*() routines which can decide the fate
#  of each part; see the mimedefang-filter man page.
#***********************************************************************
sub filter ($$$$) {
  my($entity, $fname, $ext, $type) = @_;

  return action_accept() if ($FromInternal);

  return if message_rejected(); # Avoid unnecessary work

  # Virus scan
  if ($FoundVirus) {
      my($code, $category, $action);
      $VirusScannerMessages = "";
      ($code, $category, $action) = entity_contains_virus($entity);
      if ($category eq "virus") {
        return action_quarantine($entity, "A known virus was discovered and
deleted.  Virus-scanner messages follow:\n$VirusScannerMessages\n\n");
      }
  }

  if (filter_bad_filename($entity)) {
    action_add_header("X-MIMEDefang-Status", "Quarantined: $fname");
    action_change_header("Subject", "*** Quarantined *** $Subject");
    return action_quarantine($entity, "An attachment named $fname was
removed from this document as it\nconstituted a security hazard.  If you
require this document, please contact\nthe sender and arrange an alternate
means of receiving it.\nContact the Help Desk (x2589) if you require further
assistance.\n");
  }

  # eml is bad if it's not multipart
  if (re_match($entity, '\.eml')) {
    action_add_header("X-MIMEDefang-Status", "Quarantined: $fname");
    action_change_header("Subject", "*** Quarantined *** $Subject");
    return action_quarantine($entity, "A non-multipart attachment named
$fname was removed from this document as it\nconstituted a security hazard.
If you require this document, please contact\nthe sender and arrange an
alternate means of receiving it.\nContact the Help Desk (x2589) if you
require further assistance.\n");
  }

  return action_accept();
}

#***********************************************************************
# %PROCEDURE: filter_multipart
# %ARGUMENTS:
#  entity -- a Mime::Entity object (see MIME-tools documentation for
details)
#  fname -- the suggested filename, taken from the MIME Content-Disposition:
#           header.  If no filename was suggested, then fname is ""
#  ext -- the file extension (everything from the last period in the name
#         to the end of the name, including the period.)
#  type -- the MIME type, taken from the Content-Type: header.
# %RETURNS:
#  Nothing
# %DESCRIPTION:
#  This is called for multipart "container" parts such as message/rfc822.
#  You cannot replace the body (because multipart parts have no body),
#  but you should check for bad filenames.
#***********************************************************************
sub filter_multipart ($$$$) {
  my($entity, $fname, $ext, $type) = @_;

  return action_accept() if ($FromInternal);

  if (filter_bad_filename($entity)) {
    action_add_header("X-MIMEDefang-Status", "Dropped: $fname");
    action_change_header("Subject", "*** Quarantined *** $Subject");
    action_notify_administrator("A MULTIPART attachment of type $type, named
$fname was dropped.\n");
    return action_drop_with_warning("An attachment of type $type, named
$fname was removed from this document as it\nconstituted a security hazard.
If you require this document, please contact\nthe sender and arrange an
alternate means of receiving it.\nContact the Help Desk (x2589) if you
require further assistance.\n");
  }

  # eml is bad if it's not message/rfc822
  if (re_match($entity, '\.eml') and ($type ne "message/rfc822")) {
    action_add_header("X-MIMEDefang-Status", "Dropped: $fname");
    action_change_header("Subject", "*** Quarantined *** $Subject");
    return action_drop_with_warning("A non-message/rfc822 attachment named
$fname was removed from this document as it\nconstituted a security hazard.
If you require this document, please contact\nthe sender and arrange an
alternate means of receiving it.\nContact the Help Desk (x2589) if you
require further assistance.\n");
  }

  return action_accept();
}

#***********************************************************************
# %PROCEDURE: defang_warning
# %ARGUMENTS:
#  oldfname -- the old file name of an attachment
#  fname -- the new "defanged" name
# %RETURNS:
#  A warning message
# %DESCRIPTION:
#  This function customizes the warning message when an attachment
#  is defanged.
#***********************************************************************
sub defang_warning ($$) {
  my($oldfname, $fname) = @_;
  return
    "An attachment named '$oldfname' was converted to '$fname'.\n" .
    "To recover the file, right-click on the attachment and Save As\n" .
    "'$oldfname'\n";
}

# If SpamAssassin found SPAM, append report.  We do it as a separate
# attachment of type text/plain
sub filter_end ($) {
  my($entity) = @_;

  return if ($FromInternal);

  # No sense doing any extra work
  return if message_rejected();

  # Spam checks if SpamAssassin is installed

  if ($Features{"SpamAssassin"}) {
    if (-s "./INPUTMSG" < $SizeSkip*1024) {
      # Only scan messages smaller than 256kB.  Larger messages
      # are extremely unlikely to be spam, and SpamAssassin is
      # dreadfully slow on very large messages.
      my($hits, $req, $names, $report) = spam_assassin_check();
      if ($hits >= $req) {
	my($score);
	action_add_header("X-Spam-Status", "Yes, hits=$hits req=$req");
	action_change_header("Subject", "*** SPAM *** $Subject");
	action_add_part($entity, "text/plain", "-suggest", "$report\n",
			"SpamAssassinReport.txt", "inline");

        # Clean up HTML if Anomy::HTMLCleaner is installed.
        if ($Features{"HTMLCleaner"}) {
          if ($type eq "text/html") {
            return anomy_clean_html($entity);
          }
        }
      } else {
	action_add_header("X-Spam-Status", "No, hits=$hits req=$req
$names");
      }
    }
  } else {
    action_add_header("X-Spam-Status", "No, skipped due to size");
  }
}

# DO NOT delete the next line, or Perl will complain.
1;