[Mimedefang] Greylisting++

[Cormack, Ken]

> So our (post-DATA) greylister takes into account the 4-tuple
> (sender-e-mail, recipient-e-mail, sender-ip, message-subject) for
> greylisting purposes.  Works really well!  (Alas, not patentable
> because there is prior mention of this technique elsewhere...)

I like this idea (and had also seen prior mention of the technique
elsewhere.)

I was already doing some filtering on subject-lines (see the archives), and
figured I'd break out my subject-line "normalizing" into a sub of it's own,
so that it now can be used to groom a raw $Subject into something I can
safely store in my greylist database.  I'll now just call this function when
I do my subject-line lookups, instead of doing these steps, directly in the
subject-line lookup function.

But I'll now also be able to call it, to add that fourth dimension of the
"4-tuple", in my greylisting.

It can be called with something like this, any time it's needed:

	my $normalized_subject = normalize_subject ($Subject);

	
	sub normalize_subject
	{
	  $lc_subject = lc ($Subject);
	  # Change non-alphas and non-numerics to spaces:
	  # "f_r_e_e s_t_u_f_f" becomes "f r e e s t u f f"
	  # "_free_stuff_" becomes " free stuff "
	  # "t_h_e breaking news:" becomes "t h e breaking news "
	  # "breaking.news:" becomes "breaking.news "
	  $lc_subject =~ s/[^a-z0-9]/ /g;
	
	  # remove white space from the middle so that
	  # "free s t    u f f here" becomes "free s t u f f here"
	  $lc_subject =~ s/(\s)\s+/$1/g;
	
	  # collapse "free  s t u f f  here" into "free stuff here"
	  $lc_subject =~ s!((^|\s)\S\s(\S(\s|$)){2,})!
	    my $lc_subject_x=$1;$lc_subject_x=~s/\s//g;sprintf "%s","
$lc_subject_x ";!ego;
	  $lc_subject =~ s/^\s+//;     # Trim leading whitespace
	  $lc_subject =~ s/\s+$//;     # Trim trailing whitespace
	  $lc_subject =~ s/^re //;     # Trim leading "re:"
	  $lc_subject =~ s/^fw //;     # Trim leading "fw:"
	  $lc_subject =~ s/^fwd //;    # Trim leading "fwd:"
	  $lc_subject =~ s/\s+/./g;    # Collapse whitespace into periods
	
	  $normalized_subject = $lc_subject;
	  # md_graphdefang_log ('info', "normalized_subject:
\"\<$normalized_subject\>\"");
	  return $normalized_subject;
	}

Ken