[Mimedefang] Re: MTA

[G.W. Haywood]

Hello again,

On Sun, 20 May 2007 Les Mikesell wrote:

> I like to think of spam filtering as a balance between the good of
> dropping actual spam and the collateral damage of dropping valid
> mail, slowing deliveries, and inconveniencing others with
> unnecessary delays.

You have no argument from me there, although I'd add to your list of
inconveniences that of spending hundreds of hours trying to achieve
the balance.  Most of the users I serve understand all this, they're
pleased that the effort is being made, and forgive the odd mistake as
they sometimes make mistakes themselves.  Like pulling the plug out
of the UPS with the vacuum cleaner.  Or letting a nine-year-old play
games on the fileserver in a machine shop.  (At home he'd been taught
to switch off when he'd finished, so that's what he did.)

> You presented one side of this balance as though it alone was a
> victory.  I'm just curious about the other side...

If that's the way it came over I'm sorry:

1.  I don't think this is a polarity issue, but one of many facets; I
    wasn't trying to present a side - just one take on it.

2.  I certainly don't consider the loss of billions of [currency unit]s
    in wasted time and stolen life savings to be any kind of a victory.
    The only winners in any of this are the more successful spammers,
    by which I mean to include the many ISPs who seem to tolerate spam
    if not actually to encourage it, and who make it all possible by
    failing to take the most rudimentary precautions against criminal
    use of the Internet.

3.  Not implied by your question but for the avoidance of doubt, I do
    think that the more processor-intensive measures such as ClamAV
    and MimeDefang are valuable.  It's just that in the relatively
    small installations that I run they don't have much to get their
    teeth into.  For example I used to rely a lot on SpamAssassin's
    Bayesian classifier, but there's not enough leaking through the
    other defences now to train it well.  That's partly because I've
    become, unfortunately, much more familiar with spam and spammers
    than I was a few years ago, and partly because I accept no mail
    at all from a depressingly long list of countries - which makes
    me think I should look at blocking by AS numbers.  At one site
    I see more rejections by ClamAV and MimeDefang than from all the
    rest combined.  (They have more, um, lay users than other sites
    which I work with.  If they want to send an email to Turkey they
    ask me to unblock it temporarily.)

> any evidence you might have as to why you think the balance you've
> chosen is correct - particularly if you are making these decisions
> on behalf of others as a system administrator.

In my reply to the OP I took some trouble to say that I don't think
this approach will suit everyone.  There is more 'collateral damage'
as you call it and either that's acceptable or it isn't.  It's very
time-consuming.  Is it the 'correct' approach?  I really don't know.

I don't think a system administrator should make decisions on behalf
of others, but the OP didn't ask for an opinion on that.  I think that
the task is to establish the requirements of those he serves, and then
to implement policices which, hopefully, best fit those requirements.
I communicate with the people who use the services that I provide.
(And they communicate with me... :)  It probably helps that I don't
charge for services.  If anyone thinks I'm the "Sysadmin from Hell",
I really do hope he's a spammer.

> Things like the length of a greet-pause that catches the most spam
> senders but does not interfere with valid mailers would be useful
> information for everyone.

Any greetpause can affect genuine mailers.  Whether they're correctly
implementing the standards or not probably doesn't matter, your user
may need the mail.  As I said, it may be necessary to make exceptions.
For example we have had problems with mail from mac.com now and then.
So, apparently, have others.  For now their servers see no pause here.

There are many other things in the cauldron - particularly iptables,
which in my installations stops over 99% of the spam.  I've only been
using Sendmail's greetpause for about two years, and because I'm
naturally cautious I've made only gradual and incremental changes to
the configuration.  So it's difficult to give a precise answer to your
question, but I can say that my current greetpause delays block about
55% of those attempts to connect to my mailservers which manage to get
past iptables, so that about 0.4% of spam gets through just those two
defences.  Then it has to get past a few other defences such as the
access database, greylisting, milter-regex, about half-a-dozen RBLs,
ClamAV, MimeDefang and SpamAssassin which are all individually capable
of blocking around 90% of spam.  I'm not prepared to give exact values
for greetpause just in case the wrong people are reading, but it won't
be giving away much to say that, looking at the statistics, it's clear
that most of the spammers that I see are either

1.  in a hurry or

2.  not very imaginative.

For an informative although dated discussion, see for example

http://www.issociate.de/board/post/240875/greet_pause_whitelisting.html

See also

http://lists.roaringpenguin.com/pipermail/mimedefang/2005-June/027162.html

as there's more than one way to skin this particular cat.

That's enough from me now.  I hope it helps.

--

73,
Ged.