[Mimedefang] Greylisting is GREY-ATE

[Lucas Albers]

Jonas Eckerman said:
> I don't think that'd bee too hard. I think David uses the more portable
> method with a lock file in his code. That method should work fine with my
> code as well.
I posted a reimplementation of David Skoll's original greylisting code
that solves all the lock issues with some syntax changes.
Search archives.

>
> OTH, it shouldn't be too hard extending Davids code and giving it some of
> the futures from my code either. Especially my handling of from addresses
> and IP-octets (in order to let mails from maililing lists and server parks
> thorugh should be easy to just copy to his implementation.
>
I am currently working on this, by looking at Jonas's code and applying it
to my implemntation of Dave's greylist code.

>
>>  Also, should I be worried about any pitfalls of the basic
>>  implementation by David.
>
> If Davids implementation isn't prepared for future changes, you might want
> to do something similar to the way I store the data as you are allready
> planning to add features. Note that I don't remember how he stores stuff
> in the database, so this might well be taken care of allready.


>
> I've deliberately decided on storing the stuff in a way that allows for
> future expansions and/or changes. That's why I prefix the keys with "I:"

I thought about this same issue, I currently just make another key.


> For example I'm thinking of checking how many times a host has retried
> before the black period has ended. Some hosts retries about once a minute,
> wich I'd like to punish with a slight extra delay. Some (few) hosts
> retries as fast as they can (the worst I've seen waited between 1 and 8
> seconds between each retriy, even after it got a permanent error), wich
> I'd like to punish with a longer blacklisting (measured in days or weeks
> rather than seconds or minutes). I could do this without changing much,
> adding a new field to the end of each new entry in the database.

To verify, you are only punishing sender/ip recipient pairs with this
delay, not just the ip address, correct?
Setting a 30-60 minute minimal delay on machines that do this should block
it effectively.

I saw hosts that would switch relays and reattempt delivery to the same
recipient 5-10 times in less than 15-30 seconds.
The only commonality is the helo command from the server, and that it is
delivering mail using multiple relays that are not in the same domain to
the same recipient in under 30 seconds.

>
> I'm also considering storing other stuff as well. For example I'm thinking
> of blacklisting any host that retries a mail more than "x" times after it
> was rejected with a permanent failure. This would,
> however, require that either the actual mesage or rather a hash of it
> (excluding headers maybe) is stored
What if a spammer send through a relay such as aol, then you blacklist
that relay, preventing other mail from coming through.
I thought about this idea, also.
>
> A, yes. And I'm having vague thought of implementing the relaydb-stuff
> directly in mimedefang-filter instead of calling an external application.
> If I actually do this, I can use the same database as for the greylist.
> /Jonas
> --

I have implemented something along the lines of relaydb directly in
mimedefang.
I am currently working on something similar to what you suggest.
I am not busy with anything else, as I work on this at home, so I should
have something out soon.
I have long gone past the point of diminishing returns on spam protection
for my job. Any further reduction on our mail server would not be
noticable.
It is already so astronomically low, < 2.34% spam by volume at SA score of
5. That my users would not notice anything lower.
But I notice...
I am currently working on analyzing the state of the sending relays, and
rejecting spam before content analysis, based on the sending MTA's
history.

This should have the most effect on high load sites, as content analysis
is the most expensive part.

I am enjoying learning how to code perl, and want to give back to mimedefang.
I'll post something shortly, after I dogfood it on my server.
I am still determing my expiration code.
My current idea is to run an expiration on old entries once every 1500
messages, or every 12 hours, whichever comes first.


I've been having an interesting time reading your implementation;
Getting some good ideas from it.

-- 
Luke Computer Science System Administrator
Security Administrator,College of Engineering
Montana State University-Bozeman,Montana