Folllow-up Test Code - Re: [Mimedefang] Potential for Businessmail servers to nothavereverse DNS

[John Rudd]

Jonas Eckerman wrote:
> 
> John Rudd wrote:
>>> static-70-21-118-207.res.east.verizon.net.
> 
>> So, I've been considering moving those checks to filter_end and having 
>> it generate tags that indicate the message should be treated as spam 
>> instead of being rejected.
> 
> What works fine for me is to do this checks just before calling 
> SpamAssassin and add the result as headers to the mail that SpamAssassin 
> checks.
> 
> This way I can have SpamAssassin rules scoring for things that looks 
> like dynamic or dul hostnames. So a dynamic looking host name in itself 
> won't stop a mail from beeing received, but it will increase the 
> likelyhood of the mail beeing rejected.
> 
> (This kind of thing is why I patched mimedefang.pl so I can add my own 
> headers to SpamAssassins input.)
>

That seems reasonable, but I have two other thoughts:

1) could an SA plugin, instead of external routines that need to modify 
the message itself, be used?  (when MD calls SA, is the most 
recent/current Received header already there?)  If so, that might be a 
cleaner, and more widely usable, way to do this.  (I don't think it can 
be done with regular SA rules, because I don't think there's a way to 
specify the current Received header only for a check, thus the use of a 
plug-in)

2) The environment I'm moving to will actually have 2 anti-spam engines 
(SA and Cloudmark), plus this third set of DNS checks.  I'm actually 
thinking of having their scores kept separate (instead of one feeding 
the others).  And then figuring out a way to combine them in the end. 
Something like this:

    a) For the DNS checks:
       i) if the relay has no PTR record, score = 10 or 8
      ii) if the name in the PTR record points to is a CNAME, = 8
     iii) if the name in the PTR record doesn't point back to the IP 
addr, = 8
      iv) if the name contains any of those "client" type name features, = 5

    b) Convert the cloudmark percentage to an SA type score as follows:
       Cloudmark 100 = 10, 99 = 9, 98 = 8, 97 = 7, 96 = 6, 95 = 5,
                  90-94 = 4, 80-89 = 3, 70-79 = 2, 60-69 = 1, 0-59 = 0

    c) the "combined" score starts with the highest of the 3 scores (the 
above DNS check score, the converted Cloudmark score, and the SA score). 
  If any one of them scored 10 or higher, reject the message.  If any 
two scores are 8 or higher, reject the message.  If all three scores are 
5 or higher, boost the combined score by 2*, only reject the message if 
the new score is at or above 10.  Put the combined score into two new 
headers, one descriptive (something like "Yes/No, Combined=W DNS=X 
Cloudmark=Y SpamAssassin=Z"), and one indicating score level (using "s" 
instead of "*").


(* or, maybe boost by 1 if two scores are >= 5, boost by 2 if three 
scores >= 5; or, if you're using more than 3 systems, you could extend 
this score past 2 ... boost = (# of 5+ scores) - 1  (with a minimum of 0) )

So:
   * if one score is 10+, then the message will get rejected
   * if two scores are 8+, it'll get rejected
   * if one score is 9+, and at least one other is 5+, it'll get rejected.
   * if one score is 8+, and the other two are 5+, it'll get rejected.
   * else, if any score is 5+, mark it as spam and deliver
   * else, mark it as ham and deliver

(if you're only using SA and the DNS checks, and not another anti-spam 
engine, then the 4th case doesn't apply, because you wont have two other 
5+ scores)


Which means that my DNS checks will definitely mark it as spam (and 
cause it to feed into my quarantine system), and might reject it if it 
has _no_ PTR record (if that score is 10), but it otherwise takes a 
wider agreement among my spam detectors to cause a rejection.