[Mimedefang] early experiences with grey listing

Paul Murphy pmurphy at ionixpharma.com
Thu Jan 13 05:06:08 EST 2005


> I found that I get a little nervous when mail comes in from a client,
> and I know we're going to stall it for an hour.  I just hope
> the senders' MTA will try again in a timely fashion. So far so good.
> However, if I'm waiting for a website to send me my password info.,
> I really would like the reply sooner rather than later.

The default in most implementations is one minute - the point of greylisting is
that most spam mailers have several million addresses to send to, so even if
they bother to check the return codes, most take no action on them, and skip to
the next address.  As a result, they never come back, so the greylist entry
remains grey.  Setting the delay to an hour only impacts legitimate messages
(plus spam from mailers which will try again), and since many SMTP
implementations will try again after 5, 15, 60, 120 minutes and so on, you risk
missing the first three and delaying two hours.

There are also some seriously broken mailers out there which interpret the 4xx
(temporary error) return code from greylisting as a permanent error, and never
try again.  One of the largest UK cunsumer ISPs is unfortunately one of them, so
you need a whitelist as well.  I do this using a hash lookup:

  my @senderparts = split(/@/, $sender);               
  my $senderdom = $senderparts[1];       
  $senderdom =~ s/[<>]//g;
  if (
       (exists $IncomingWhitelist{lc($sender)}) ||
       (exists $IncomingWhitelist{lc($senderdom)}) ||
       ( exists $LocalRelayAddrs{$ip}) )     
    md_syslog('info',"WhiteList - skip greylisting check,
Sender=$sender,Dom=$senderdom, Addr=$ip");
    return ('CONTINUE', "ok");                 

    #Check greylist
    if (defined($greylist) && $greylist && !(greylist_ip_whitelist($ip)) &&
                      !check_authenticated() )
      $grey = greylist_check($ip,$sender,$recipient);

My whitelist entries are defined at the top of the filter:

# Addresses to allow regardless of origin
$IncomingWhitelist{'btopenworld.com'} = 1;
$IncomingWhitelist{'friend at spammer.com'} = 1;

> I'm thinking that for our set up, the more appropriate thing to
> do might be to try and verify that the sender's domain reverse
> maps to the class C sending IP address.  If it does then let it
> through without delay.  After all, SA is still available
> to scan the message, and most spam uses zombies and/or address spoofs.

Not necessarily true - I've had issues with spam from domains which appear to be
legitimate, and in large quantities.  Even though SA scores them between 8 and
25, and we bounce anything over 10, their persistence made it a nuisance, and I
ended up blocking their IP addresses at the firewall level.

> I'm also thinking it would be worthwhile whitelisting any
> recipients of mail originated from the local/trusted network. Thus,
> if a user initiates a contact, there is some assurance that he'll
> see the reply without delay.

Yes, that's an area which the current implementation doesn't do, and which would
be reasonably easy to add.  However, be wary of auto-replies - if a spammer
sends a message to someone who is out of the office, and they reply
automatically, your database now lists the spammer as clean.  In most cases, the
database gets to a fiarly stable state after a couple of days, since the
important correspondents will mail you regularly.  Even those who don't
shouldn't suffer more than a 10 minute delay at worst if their mailer is
> The other thing that I've noticed is a lot of spam sends to local
> mail addresses that don't exist.  With greylisting enabled, these
> errant messages are first tempfail'ed, and only if/when they come
> back after the black out period, is the recipient id checked,
> and the sender is then bounced with "unkown user".  Not necessarily
> good, if a sender mistypes a local email address, and would like to
> know about it soonder rather than later.  I don't know this authentication
> can be implemented in the MIMEDefang filter or not, with some reasonable
> level of effort, but it seems like it might be a good thing to do.

It is already available using md_check_against_smtp_server() in your filter if
that is supported (and note that Exchange 2000 is broken, so doesn't work), and
you can also use LDAP to do real-time queries against AD or any other
LDPA-compatible directory system.  Alternatively, some people harvest valid
addresses from their systems into a local DB file daily, and check that from MD.

Again, if your grey period was reduced to a minute, senders who mistype the
address will find out faster.  Also, since the greylist check is in
filter_recipient, you can do the address validation check before the greylist
check, and bounce early.

Best Wishes,

Paul Murphy
Head of Informatics
Ionix Pharmaceuticals Ltd
418 Science Park, Cambridge, CB4 0PA

Tel. 01223 433741
Fax. 01223 433788

This email and any files transmitted with it are confidential and 
intended solely for the use of the individual or entity to which they
are addressed.  If you have received this email in error please contact
the sender or the Ionix IT Helpdesk on +44 (0) 1223 433741

More information about the MIMEDefang mailing list