[Mimedefang] sendmail and filter_helo interaction

[Dirk the Daring]

    Jim McCullars and I have been discussing filter_helo offlist, and 
David's observation (supported by Jim's experimentation) that if 
filter_helo returns a REJECT, the connection is not immediately rejected, 
but rather is rejected after MAIL FROM.

    It happens that I have been using a heavily-logged filter, and examined 
some test connections that Jim made to my mailserver. Some interesting 
results came out of that. I'm using sendmail v8.13.8 and MIMEDefang v2.57 
on Solaris 9 with Perl v5.8.6.

    It is true that even if filter_helo returns REJECT, the connection is 
not immediately dropped. The sending host can still issue another command, 
such as MAIL FROM.

    However, it is also true that if the sending host does issue another 
command, like MAIL FROM, that there is no corresponding MILTER call. 
MIMEDefang never sees a call to filter_sender if filter_helo returns a 
REJECT.

    So it *appears* like the connection is maintained, but it also seems 
that the SMTP conversation *effectively ends* if filter_helo returns 
REJECT. The connecting host can issue another command, but it will be 
ignored.

    I've theorized that if the connecting host issues a RSET followed by 
another (valid) HELO, the connection can proceed and be successful. This 
might be why the connection is not immediately dropped. Also, I use 
FEATURE(`delay_checks'), which may have something to do with it.

    Something else also came out of the experiments. If a connecting host 
violates GREETPAUSE (sends before presentation of the banner), its HELO is 
still passed via MILTER call to MIMEDefang.

    However, if RATECONTROL is tripped, there is no MILTER call. Presumbly, 
this is true of CONNCONTROL.

    Finally, I got asked about my filter_helo code. My current filter has 
a *lot* of logging statements, because I've been experimenting and need 
to meter its function. I'm also not a Perl hacker, so I doubt my code 
is the most-efficient way to write this. Here it is...feel free to 
use/adapt the code as may suit your needs, or ignore my code and write 
your own:

# Some global variables used by the filter* functions
###############################
# Declare a hash of
#       - Key: IP addresses we consider internal
#       - Value: Flag as to if host is exempt from AV scans (0=No, 1=Yes)
###############################
%OurHosts=(     "127.0.0.1", 0,
                 "192.168.2.2", 0,
                 "10.2.3.4", 0 );

###############################
# Declare a hash of
#       - Key: Domain Names we host
#       - Value: A flag as to if the Domain should be
#                       receiving E-Mail (0=No, 1=Yes)
###############################
%OurDomains=(   "mydomain.tld", 1,
                 "otherdomain.tld", 1,
                 "notadomain.tld", 0 );

[..other code..]

#***********************************************************************
# %PROCEDURE: filter_helo
# %ARGUMENTS:
#  IP address of remote host; hostname of remote host; HELO string
#  presented by remote host
# %RETURNS:
#  2-5 element array (see documentation)
# %DESCRIPTION:
#  Called after SMTP connection has been established and sending host has
#  given a HELO statement, but not MAIL FROM: or RCPT TO:
#***********************************************************************
sub filter_helo ()
 	{
 	# Read the parameters passed to the function
 	my($hostip, $hostname, $helo) = @_;

 	# Local string for Domain name processing
 	my($domainstring);
 	# Local variable for string indexing
 	my($subindex)=0;

 	# Search the list of our hosts using the $hostip argument
 	if ( exists($OurHosts{$hostip}) )
 		{
 		# Recognize our internal host
 		md_syslog('info', "Internal Host $hostip HELO $helo");
 		# Don't look at it further
 		return('CONTINUE', 'ok');
 		}
 	else
 		{
 		# Foreign host
 		md_syslog('info', "Foreign Host $hostip HELO $helo");
 		}

 	# Check if the HELO is an IP address
 	if ($helo =~ /^(\[?)(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})(\]?)$/ )
 		{
 		# HELO looks like an IP - the comparison will split the string
 		#       into 3 variables; $1 will have [ or be undefined, $2 will
 		#       have the IP address without any brackets, $3 will have
 		#       ] or be undefined ($1 and $3 are undefined if HELO lacked
 		#       square brackets)
 		md_syslog('info', "IP HELO $helo");

 		# Check #0
 		# The IP address portion should *not* be identical to the
 		#       original HELO string - if it is, the original HELO lacked
 		#       brackets and therefore is invalid (this is "safer" than
 		#       trying to evaluate $1 and $3 directly, as they may be
 		#       undefined, or have garbage from a previous iteration)
 		if ( $2 eq $helo )
 			{
 			# Reject connection - invalid HELO
 			md_syslog('alert', "Invalid HELO $helo by Host $hostip");
 			return('REJECT', "INVALID HELO/EHLO: $helo is not valid");
 			}

 		# Check #1
 		# Since the HELO was an IP address, it should match the host's IP
 		if ( $2 ne $hostip )
 			{
 			# Reject connection - fraudulent HELO
 			md_syslog('alert', "Fraudulent HELO $helo by Host $hostip");
 			return('REJECT', "FRAUDULENT HELO/EHLO: $helo is not $hostip");
 			}
 		}
 	else
 		{
 		# HELO looks like a host name string
 		md_syslog('info', "Non-IP HELO $helo");

 		# Check #2
 		# If the HELO is a Domain Name, it will contain at least one "."
 		if ( index($helo, ".") == -1 )
 			{
 			# Reject connection - invalid HELO
 			md_syslog('alert', "Invalid HELO $helo by Host $hostip");
 			return('REJECT', "INVALID HELO/EHLO: $helo is not valid");
 			}

 		# Check #3
 		# HELO should not contain "localhost"
 		if ( $helo =~ /localhost/i )
 			{
 			# Reject connection - invalid HELO
 			md_syslog('alert', "Invalid HELO $helo by Host $hostip");
 			return('REJECT', "INVALID HELO/EHLO: $helo is not valid");
 			}

 		# Check #4
 		# If the HELO is an FQDN, the index and rindex of "." will not be the same
 		# This catches the spammer using domain.tld (which will slip
 		#       by Check #2)
 		if ( index($helo, ".") == rindex($helo, ".") )
 			{
 			# Reject connection - invalid HELO
 			md_syslog('alert', "Non-FQDN HELO $helo by Host $hostip");
                         return('REJECT', "INVALID HELO/EHLO: $helo is not FQDN");
 			}

 		# Check #5
 		# HELO should not be a hosted Domain
 		$domainstring=$helo;
 		while ( index($domainstring, ".") != rindex($domainstring, ".") )
 			{
 			# Extract the substring of text after (+1) the first "."
 			$subindex=(index($domainstring, ".")) + 1;
 			$domainstring=substr($domainstring, $subindex);
 			if ( exists($OurDomains{$domainstring}) )
 				{
 				# Reject connection - fraudulent HELO
 				md_syslog('alert', "Fraudulent HELO $helo by Host $hostip");
 				return('REJECT', "FRAUDULENT HELO/EHLO: $hostip is not $helo");
 				}
 			}
 			# END OF WHILE

 		# If we reach here, the loop has ended due to the $helo string being
 		# pared down to just Domain and TLD, without a match - the HELO is
 		# not trying to impersonate a hosted Domain
                 }
                 # End of IF

 	# At this point, we've eliminated all the obvious fraudulent HELOs

 	# HELO has passed all checks
 	md_syslog('alert', "Accepted HELO $helo by Foreign Host $hostip");
 	return('CONTINUE', 'ok');
 	}
 	# End of filter_helo

    As I wrote previously, my entire filter is heavily logged. My analysis 
of those logs indicates that only about 50% of foreign mailhosts 
connecting to my network get past HELO. Based on the I-think-reasonable 
assumption that no "legitimate" mail server would be tripped up by 
GREETPAUSE, RATECONTROL, CONNCONTROL or the tests I have in filter_helo, 
my conclusion is that those 50% are spammers, and I'm effectively stopping 
them by the end of HELO.