[Mimedefang] Sender validation

[Jonas Eckerman]

The idea of validating addresses in MAIL FROM with MX servers has been around for sometime, but has some problems. I'm currently evaluating a pretty involved validation scheme, and it does seem to work out so far.

If anyone sees anything obviously stupid in the snippets below, please tell me. If this stuff keeps working without problems, I'll post again if I actually start rejecting based on it. It does add some time, but I like almost anything that rejects before I call SA.

Here's some snippets:

The actual check. Note that we do not just see any REJECT return from md_check_against_smtp_server as a reason to reject.
--8<--
# Check a mail address against a mail server
sub CheckMailAddress($$) {
	my ($a,$s) = @_;
	my ($ok,$msg,$code,$dsn) = md_check_against_smtp_server('<>',$a,$MyFilterHostName,$s);
	my $txt = "$code $dsn $msg";
	$txt =~ s/\s\s+/ /g;
	$txt =~ s/^\s+//;
	$txt =~ s/\s+$//;
	# Disregard some REJECTs because they aren't really rejecting the recipient address.
	return (3,$txt) if ($ok eq 'REJECT' && ($code !~ /^55[0134]$/ || $msg =~ /(sender|mail from|return|<>)/i));
	return (1,'') if ($ok eq 'CONTINUE');
	return (0,$txt) if ($ok eq 'REJECT');
	return (2,'');
}
--8<--

The MX checking stuff. We loop through MX's and only take action if we get a clear reject or go ahead from the above sub. This means we sometimes get a go ahead from a secondary server, but I'd rather let some bad senders thorugh that reject some good ones.
--8<--
# Check a mail address against it's MX server(s)
sub CheckMailAddressMX($) {
	my ($a) = @_;
	return (4,'') if ($a =~ /^<?>?$/);
	my $d = $a;
	$d =~ s/^.*@([^@>]*)>?$/$1/;
	return (5,'') if (!$d);
	my $dns = Net::DNS::Resolver->new;
	$dns->defnames(0); # do not search default domain
	$dns->persistent_tcp(0);
	$dns->tcp_timeout(15);
	#$dns->udp_timeout(15);
	my $mx = $dns->query($d, 'MX');
	return (6,'') if (!$mx);
	my %mx;
	foreach my $r ($mx->answer) {
		$mx{$r->preference} = $r->exchange if ($r->type eq 'MX');
	}
	return (7,'') if (!%mx);
	my @rinfs = ();
	foreach my $mp (sort keys %mx) {
		my ($ok,$rinf) = CheckMailAddress($a,$mx{$mp});
		return (0,$rinf) if (!$ok);
		return (1,$rinf) if ($ok == 1);
		push @rinfs, $rinf if ($rinf);
	}
	return (8,join('; ', at rinfs));
}
--8<--

A snippet from "filter_sender". As you can see, some addresses are excempted from the check, and local domains are checked without MX lookups. Again, I'd rather let some bad senders thorugh that reject some good ones. I'm also trying not to hit list servers. I'm still monitoring results, so I might add some stuff to excempt more addresses. *.frukt.org is never checked since it has a wildcard MX (it is for a FNT gateway).
--8<--
	# Check if sender address is valid. Exempt a bunch of addresses from the check
	# in order to be less abusive with regards to big list servers and that sort
	# of stuff.
	# Just test for now, to see if it's a bad idea.
	# If from some of our local domains, check locally.
	if ($sender !~ /^<?>?$/ && $sender !~ /^<?(postmaster|abuse)@/i && $sender !~ /^<?(|.*[-_+=])(daemon|gateway)(|[-_+=].*)@/i &&
			$sender !~ /@(|[^@]+\.)(bounces|returns|lists|newsletters?)\.[^@\.]+\.[^@\.]+[^@]*$/i &&
			($sender !~ /^<?(|.*[-_+=])(anonymous|undisclosed|unspecified|list|return|users|bounces|\d+)(|[-_+=].*)@/i ||
			$sender !~ /^<?(|.*[-_+=])$OurDomains(|[-_+=].*)@/i)) {
		if ($sender =~ /^.+\@$OurDomains>?$/i && $sender !~ /^.*@(|[^@]\.)frukt.org>?$/i) {
			my ($ok,$rinf) = CheckMailAddress($sender,'127.0.0.1');
			debug_log(0,"filter_sender: $sender = $ok (local) [$rinf]");
			if (!$ok) {
				#md_syslog('info',"MDLOG,$MsgID,bad_sender,local,$ip,$sender,?,?");
				#return ('REJECT',"Bad sender address: $sender! Responsible server said: $rinf");
			}
		} elsif ($sender !~ /^.+\@$OurDomains>?$/i) {
			my ($ok,$rinf) = CheckMailAddressMX($sender);
			debug_log(0,"filter_sender: $sender = $ok (MX) [$rinf]");
			if (!$ok) {
				#md_syslog('info',"MDLOG,$MsgID,bad_sender,mx,$ip,$sender,?,?");
				#return ('REJECT',"Bad sender address: $sender! Responsible server(s) said: $rinf");
			}
		} else {
			debug_log(0,"filter_sender: $sender (unchecked 2)");
		}
	} else {
		debug_log(0,"filter_sender: $sender (unchecked 1)");
	}
--8<--

Even with the "rather go ahead than reject too much" philosophy and the excempted stuff, this would reject quite a lot of stuff. So far the stuff above hasn't hit any legit mail.

Regards
/Jonas

PS. As a curiosity I've also noticed that some spammers use domains for wich there're no MX servers actually accepting mail (there are MX records in the DNS, but the servers they point to doesn't accept mail for the domain).

-- 
Jonas Eckerman, jonas at truls.org
http://www.truls.org/