[Mimedefang] Thoughts On Filter

[jimmydf at hushmail.com]

Hello All,

I've been working on a custom mimedefang filter over the last few months,
 the requirements have changed from when I first set out to implement
a filter for attachments and spam. In the last while I have tried to
incorporate some content filtering and domain check (competitors).

Attached is my filter that seems to work well, what I have noticed is
that if I send a bad attachement through, the filter responds as expected.
However if I send the same email through the filter but to more then
one domain the attachment does not get removed, I'm hoping someone on
this list can point me in the right direction.

James.



# -*- Perl -*-
#***********************************************************************
#
# mimedefang-filter
#
# Suggested minimum-protection filter for Microsoft Windows clients,
plus
# SpamAssassin checks if SpamAssassin is installed.
#
# Copyright (C) 2002 Roaring Penguin Software Inc.
#
# This program may be distributed under the terms of the GNU General
# Public License, Version 2, or (at your option) any later version.
#
# $Id: suggested-minimum-filter-for-windows-clients,v 1.68 2003/08/08
18:28:49 dfs Exp $
#***********************************************************************

#***********************************************************************
# Set administrator's e-mail address here.  The administrator receives
# quarantine messages and is listed as the contact for site-wide
# MIMEDefang policy.  A good example would be 'defang-admin at mydomain.com'
#***********************************************************************
$AdminAddress = 'jimmy at mydomain.com.au';
$AdminName = "Sydney Systems";

#***********************************************************************
# Set the e-mail address from which MIMEDefang quarantine warnings and
# user notifications appear to come.  A good example would be
# 'mimedefang at mydomain.com'.  Make sure to have an alias for this
# address if you want replies to it to work.
#***********************************************************************
$DaemonAddress = 'MD at mydomain.com.au';

#***********************************************************************
# If you set $AddWarningsInline to 1, then MIMEDefang tries *very* hard
# to add warnings directly in the message body (text or html) rather
# than adding a separate "WARNING.TXT" MIME part.  If the message
# has no text or html part, then a separate MIME part is still used.
#***********************************************************************
$AddWarningsInline = 0;

#***********************************************************************
# To enable syslogging of virus and spam activity, add the following
# to the filter:
# md_graphdefang_log_enable();
# You may optionally provide a syslogging facility by passing an
# argument such as:  md_graphdefang_log_enable('local4');  If you do
this, be
# sure to setup the new syslog facility (probably in /etc/syslog.conf).
# An optional second argument causes a line of output to be produced
# for each recipient (if it is 1), or only a single summary line
# for all recipients (if it is 0.)  The default is 1.
# Comment this line out to disable logging.
#***********************************************************************
md_graphdefang_log_enable('mail', 1);

#***********************************************************************
# Uncomment this to block messages with more than 50 parts.  This will
# *NOT* work unless you're using Roaring Penguin's patched version
# of MIME tools, version MIME-tools-5.411a-RP-Patched-02 or later.
#
# WARNING: DO NOT SET THIS VARIABLE unless you're using at least
# MIME-tools-5.411a-RP-Patched-02; otherwise, your filter will fail.
#***********************************************************************
# $MaxMIMEParts = 50;

#***********************************************************************
# Set various stupid things your mail client does below.
#***********************************************************************

# Set the next one if your mail client cannot handle nested multipart
# messages.  DO NOT set this lightly; it will cause action_add_part to
# work rather strangely.  Leave it at zero, even for MS Outlook, unless
# you have serious problems.
$Stupidity{"flatten"} = 0;

# Set the next one if your mail client cannot handle multiple "inline"
# parts.
$Stupidity{"NoMultipleInlines"} = 0;

# The next lines force SpamAssassin modules to be loaded and rules
# to be compiled immediately.  This may improve performance on busy
# mail servers.  Comment the lines out if you don't like them.
if ($Features{"SpamAssassin"}) {
    spam_assassin_init()->compile_now(1) if defined(spam_assassin_init());

    # If you want to use auto-whitelisting:
#   if (defined($SASpamTester)) {
#       use Mail::SpamAssassin::DBBasedAddrList;
#       my $awl = Mail::SpamAssassin::DBBasedAddrList->new();
#       $SASpamTester->set_persistent_address_list_factory($awl) if defined($awl);
#   }
}

# Set networks that are trusted, ie ones that we don't want some rules
to work on
# such as spam filtering of internal mail

sub relayIsTrusted($) {

    my ($address) =@_;

    # Hash that defines a list of hosts/networks that are considered
trusted
    # to specify a single host use 255.255.255.255 as the netmask

    my %trustedSubnets = (
    '127.0.0.1'  =>  '255.255.255.255',
    '10.9.1.62'  =>  '255.255.255.255',
    );

    my $trustedRelay = 0;
    my $addr = inet_aton $address;

    while (my ($networkString, $netmaskString) = each %trustedSubnets)
{
        my $network = inet_aton $networkString;
        my $netmask = inet_aton $netmaskString;

        if (($addr & $netmask) eq $network){
            $trustedRelay = 1;
            last;
        }
     }

     return $trustedRelay;
    
} 


# This procedure returns true for entities with bad filenames.
sub filter_bad_filename ($) {

if (!relayIsTrusted($RelayAddr)) {

    my($entity) = @_;
    my($bad_exts, $re);

    # Bad extensions
    $bad_exts = '(ade|adp|app|asd|asf|asx|avi|mpeg|bas|bat|chm|cmd|com|cpl|crt|dll|exe|fxp|hlp|hta|hto|inf|ini|ins|isp|jpg|jpeg|jse?|lib||mde|msc|mov|msi|msp|mp3|swf|mst|ocx|pcd|pif|prg|reg|scr|sct|sh|shb|shs|sys|url|vb|vbe|vbs|vcs|vxd|wmd|wms|wmz|wsc|wsf|wsh|\{)';


    # Do not allow:
    # - CLSIDs  {foobarbaz}
    # - bad extensions (possibly with trailing dots) at end or
    #   followed by non-alphanum
    $re = '\.' . $bad_exts . '\.*([^-A-Za-z0-9_.,]|$)';
    return re_match($entity, $re);

} #End relayIsTrusted
}

# Scan for a virus using the first supported virus scanner we find.
sub message_contains_virus () {
    return message_contains_virus_avp()      if ($Features{'Virus:AVP'});
    return message_contains_virus_fprot()    if ($Features{'Virus:FPROT'});
    return message_contains_virus_fsav()     if ($Features{'Virus:FSAV'});
    return message_contains_virus_hbedv()    if ($Features{'Virus:HBEDV'});
    return message_contains_virus_nai()      if ($Features{'Virus:NAI'});
    return message_contains_virus_bdc()      if ($Features{'Virus:BDC'});
    return message_contains_virus_nvcc()     if ($Features{'Virus:NVCC'});
    return message_contains_virus_rav()      if ($Features{'Virus:RAV'});
    return message_contains_virus_sophie()   if ($Features{'Virus:SOPHIE'});
    return message_contains_virus_trophie()  if ($Features{'Virus:TROPHIE'});
    return message_contains_virus_sophos()   if ($Features{'Virus:SOPHOS'});
    return message_contains_virus_trend()    if ($Features{'Virus:TREND'});
    return message_contains_virus_filescan() if ($Features{'Virus:FileScan'});
    return message_contains_virus_clamd()    if ($Features{'Virus:CLAMD'});
    return message_contains_virus_clamav()   if ($Features{'Virus:CLAMAV'});
    return message_contains_virus_carrier_scan() if ($Features{'Virus:SymantecCSS'});
    return (wantarray ? (0, 'ok', 'ok') : 0);
}

# Scan for a virus using the first supported virus scanner we find.
sub entity_contains_virus ($) {
    my($e) = @_;
    return entity_contains_virus_avp($e)      if ($Features{'Virus:AVP'});
    return entity_contains_virus_fprot($e)    if ($Features{'Virus:FPROT'});
    return entity_contains_virus_fsav($e)     if ($Features{'Virus:FSAV'});
    return entity_contains_virus_hbedv($e)    if ($Features{'Virus:HBEDV'});
    return entity_contains_virus_nai($e)      if ($Features{'Virus:NAI'});
    return entity_contains_virus_bdc($e)      if ($Features{'Virus:BDC'});
    return entity_contains_virus_nvcc($e)     if ($Features{'Virus:NVCC'});
    return entity_contains_virus_rav($e)      if ($Features{'Virus:RAV'});
    return entity_contains_virus_sophie($e)   if ($Features{'Virus:SOPHIE'});
    return entity_contains_virus_trophie($e)  if ($Features{'Virus:TROPHIE'});
    return entity_contains_virus_sophos($e)   if ($Features{'Virus:SOPHOS'});
    return entity_contains_virus_trend($e)    if ($Features{'Virus:TREND'});
    return entity_contains_virus_filescan($e) if ($Features{'Virus:FileScan'});
    return entity_contains_virus_clamd($e)    if ($Features{'Virus:CLAMD'});
    return entity_contains_virus_clamav($e)   if ($Features{'Virus:CLAMAV'});
    return entity_contains_virus_carrier_scan($e) if ($Features{'Virus:SymantecCSS'});
    return (wantarray ? (0, 'ok', 'ok') : 0);
}

#***********************************************************************
# %PROCEDURE: filter_begin
# %ARGUMENTS:
#  None
# %RETURNS:
#  Nothing
# %DESCRIPTION:
#  Called just before e-mail parts are processed
#***********************************************************************
sub filter_begin () {

    # Check text file for strings we want to monitor
    # against subject and body of email
 
    if(dirty_word('subject','/etc/mail/badword')) {
        add_recipient('inoutmail at mydomain.com.au');
    }
    if(dirty_word('body','/etc/mail/badword')) { 
     add_recipient('inoutmail at mydomain.com.au');
    }
    
    if (stream_by_domain()) {
        return;
    }

    

     if (!relayIsTrusted($RelayAddr)) {

    # ALWAYS drop messages with suspicious chars in headers
    if ($SuspiciousCharsInHeaders) {
        md_graphdefang_log('suspicious_chars');
 action_quarantine_entire_message("Message quarantined because of suspicious
characters in headers");
 # Do NOT allow message to reach recipient(s)
 return action_discard();
    }

    # Scan for viruses if any virus-scanners are installed
    my($code, $category, $action) = message_contains_virus();

    # Lower level of paranoia - only looks for actual viruses
    $FoundVirus = ($category eq "virus");

    # Higher level of paranoia - takes care of "suspicious" objects
    # $FoundVirus = ($action eq "quarantine");

    if ($action eq "tempfail") {
 action_tempfail("Problem running virus-scanner");
 md_syslog('warning', "Problem running virus scanner: code=$code, category=$category,
 action=$action");
    }


     } # End of relayIsTrusted
}

#***********************************************************************
# %PROCEDURE: filter
# %ARGUMENTS:
#  entity -- a Mime::Entity object (see MIME-tools documentation for
details)
#  fname -- the suggested filename, taken from the MIME Content-Disposition:
#           header.  If no filename was suggested, then fname is ""
#  ext -- the file extension (everything from the last period in the
name
#         to the end of the name, including the period.)
#  type -- the MIME type, taken from the Content-Type: header.
#
#  NOTE: There are two likely and one unlikely place for a filename to
#  appear in a MIME message:  In Content-Disposition: filename, in
#  Content-Type: name, and in Content-Description.  If you are paranoid,

#  you will use the re_match and re_match_ext functions, which return
true
#  if ANY of these possibilities match.  re_match checks the whole name;
#  re_match_ext checks the extension.  See the sample filter below for
usage.
# %RETURNS:
#  Nothing
# %DESCRIPTION:
#  This function is called once for each part of a MIME message.
#  There are many action_*() routines which can decide the fate
#  of each part; see the mimedefang-filter man page.
#***********************************************************************
sub filter ($$$$) {

   if (!relayIsTrusted($RelayAddr)) {

    my($entity, $fname, $ext, $type) = @_;

    return if message_rejected(); # Avoid unnecessary work

    # Block message/partial parts
    if (lc($type) eq "message/partial") {
        md_graphdefang_log('message/partial');
 action_bounce("MIME type message/partial not accepted here");
 return action_discard();
    }

    # Virus scan
    if ($FoundVirus) {
 my($code, $category, $action);
 $VirusScannerMessages = "";
 ($code, $category, $action) = entity_contains_virus($entity);
 # If you are more paranoid, change to: if ($action eq "quarantine")
{
 if ($category eq "virus") {
            md_graphdefang_log('virus',$VirusName, $RelayAddr);

     # Bounce the mail!
     action_bounce("Virus $VirusName found in mail - rejected");

     # But quarantine the part for examination later.  Comment
     # the next line out if you don't want to bother.
     action_quarantine($entity, "A known virus was discovered and deleted.
 Virus-scanner messages follow:\n$VirusScannerMessages\n\n");

     return;
 }
 if ($action eq "tempfail") {
     action_tempfail("Problem running virus-scanner");
     md_syslog('warning', "Problem running virus scanner: code=$code,
 category=$category, action=$action");
 }
    }

    if (filter_bad_filename($entity)) {
        md_graphdefang_log('bad_filename', $fname, $type);
        #action_quarantine_entire_message("Message quarantined because
of suspicious characters in headers"); 
 print STDERR "Domain = $tdomain; bad_exts = $bad_exts\n";

 action_notify_sender("An attachment of type $type, named $fname was
blocked\n\n");
 return action_quarantine($entity, "An attachment named $fname was removed
from this document as it\nconstituted a security hazard.  If you require
this document, please contact\nthe sender and arrange an alternate means
of receiving it.\n");
 }

    # eml is bad if it's not multipart
    if (re_match($entity, '\.eml')) {
        md_graphdefang_log('non_multipart');
 return action_quarantine($entity, "A non-multipart attachment named
$fname was removed from this document as it\nconstituted a security hazard.
 If you require this document, please contact\nthe sender and arrange
an alternate means of receiving it.\n");
    }
    # Clean up HTML if Anomy::HTMLCleaner is installed.
    if ($Features{"HTMLCleaner"}) {
 if ($type eq "text/html") {
     return anomy_clean_html($entity);
 }
    }

    return action_accept();

     } # end of relayIsTrusted
}

#***********************************************************************
# %PROCEDURE: filter_multipart
# %ARGUMENTS:
#  entity -- a Mime::Entity object (see MIME-tools documentation for
details)
#  fname -- the suggested filename, taken from the MIME Content-Disposition:
#           header.  If no filename was suggested, then fname is ""
#  ext -- the file extension (everything from the last period in the
name
#         to the end of the name, including the period.)
#  type -- the MIME type, taken from the Content-Type: header.
# %RETURNS:
#  Nothing
# %DESCRIPTION:
#  This is called for multipart "container" parts such as message/rfc822.
#  You cannot replace the body (because multipart parts have no body),

#  but you should check for bad filenames.
#***********************************************************************
sub filter_multipart ($$$$) {

    if (!relayIsTrusted($RelayAddr)) {

    my($entity, $fname, $ext, $type) = @_;

    if (filter_bad_filename($entity)) {
        md_graphdefang_log('bad_filename', $fname, $type);
 action_notify_administrator("A MULTIPART attachment of type $type, named
$fname was dropped.\n");
 return action_drop_with_warning("An attachment of type $type, named
$fname was removed from this document as it\nconstituted a security hazard.
 If you require this document, please contact\nthe sender and arrange
an alternate means of receiving it.\n");
    }

    # eml is bad if it's not message/rfc822
    if (re_match($entity, '\.eml') and ($type ne "message/rfc822")) {
        md_graphdefang_log('non_rfc822',$fname);
 return action_drop_with_warning("A non-message/rfc822 attachment named
$fname was removed from this document as it\nconstituted a security hazard.
 If you require this document, please contact\nthe sender and arrange
an alternate means of receiving it.\n");
    }

    # Block message/partial parts
    if (lc($type) eq "message/partial") {
        md_graphdefang_log('message/partial');
 action_bounce("MIME type message/partial not accepted here");
 return;
    }

    return action_accept();

    }# end of if relayIsTrusted
}


#***********************************************************************
# %PROCEDURE: defang_warning
# %ARGUMENTS:
#  oldfname -- the old file name of an attachment
#  fname -- the new "defanged" name
# %RETURNS:
#  A warning message
# %DESCRIPTION:
#  This function customizes the warning message when an attachment
#  is defanged.
#***********************************************************************
sub defang_warning ($$) {
    my($oldfname, $fname) = @_;
    return
 "An attachment named '$oldfname' was converted to '$fname'.\n" .
 "To recover the file, right-click on the attachment and Save As\n" .
 "'$oldfname'\n";
}

# If SpamAssassin found SPAM, append report.  We do it as a separate
# attachment of type text/plain
sub filter_end ($) {


    $UCSender=uc($Sender);

    my (@sendarry, $spysender);
    open (DATA, " < /etc/mail/senderfile");
    while (<DATA>) {
       $_="\U$_";
       chomp;
       push (@sendarray,$_);
       }

    foreach $spysender(@sendarray){
       if ($UCSender eq "$spysender"){
 add_recipient('inmail at mydomain.com.au');
 }
    }
    
    $UCDomain=uc($Domain);
    my (@domainarray, $spydomain);
    open (DATA, " < /etc/mail/domainfile");
    while (<DATA>) {
 $_="\U$_";
        chomp;
 push (@domainarray,$_);
 }

    foreach $spydomain(@domainarray){

 if ($UCDomain eq "$spydomain") {
 add_recipient('outmail at mydomain.com.au');
        
 }
    }

    my (@ext_sender, $extsender);
    open (DATA, " < /etc/mail/externalfile");
    while (<DATA>) {
        $_="\U$_";
 chomp;
 push (@ext_sender,$_);
 }
    
    foreach $extsender(@ext_sender){
        if ($UCSender =~ m/$extsender/){
 add_recipient('inmail at mydomain.com.au');

        }
    }
    my($entity) = @_;

    # If you want quarantine reports, uncomment next line
    send_quarantine_notifications();

    # IMPORTANT NOTE:  YOU MUST CALL send_quarantine_notifications()
AFTER
    # ANY PARTS HAVE BEEN QUARANTINEliD.  SO IF YOU MODIFY THIS FILTER
TO
    # QUARANTINE SPAM, REWORK THE LOGIC TO CALL send_quarantine_notifications()
    # AT THE END!!!

    # No sense doing any extra work
    #return if message_rejected();

    if (!relayIsTrusted($RelayAddr)) {

    # Spam checks if SpamAssassin is installed
    if ($Features{"SpamAssassin"}) {
 if (-s "./INPUTMSG" < 100*1024) {
     # Only scan messages smaller than 100kB.  Larger messages
     # are extremely unlikely to be spam, and SpamAssassin is
     # dreadfully slow on very large messages.
     my($hits, $req, $names, $report) = spam_assassin_check();
     my($score);
     if ($hits < 40) {
  $score = "*" x int($hits);
     } else {
  $score = "*" x 40;
     }
     # We add a header which looks like this:
     # X-Spam-Score: 6.8 (******) NAME_OF_TEST,NAME_OF_TEST
     # The number of asterisks in parens is the integer part
     # of the spam score clamped to a maximum of 40.
     # MUA filters can easily be written to trigger on a
     # minimum number of asterisks...
     action_change_header("X-Spam-Score", "$hits ($score) $names");
            #action_change_header("Subject", "***SPAM*** $Subject");
     if ($hits >= $req) {
                md_graphdefang_log('spam', $hits, $RelayAddr);

  # If you find the SA report useful, add it, I guess...
 # action_add_part($entity, "text/plain", "-suggest",
 #                 "$report\n",
 #   "SpamAssassinReport.txt", "inline");
     } else {
  # Delete any existing X-Spam-Score header?
  action_delete_header("X-Spam-Score");
     }
 }
    }

    # I HATE HTML MAIL!  If there's a multipart/alternative with both
    # text/plain and text/html parts, nuke the text/html.  Thanks for
    # wasting our disk space and bandwidth...

    # If you don't mind HTML mail, comment out the next line.
    remove_redundant_html_parts($entity);

    } # end if relayIsTrusted

}

# DO NOT delete the next line, or Perl will complain.
1;




Concerned about your privacy? Follow this link to get
FREE encrypted email: https://www.hushmail.com/?l=2

Free, ultra-private instant messaging with Hush Messenger
https://www.hushmail.com/services.php?subloc=messenger&l=434

Promote security and make money with the Hushmail Affiliate Program: 
https://www.hushmail.com/about.php?subloc=affiliate&l=427