HTML e-mail is unspeakably evil (was Re: [Mimedefang] Considering an additional spam filter)

[Jeffrey Goldberg]

On Mon, 26 May 2003, David F. Skoll wrote:

> HTML mail is unspeakably evil, and simple-minded solutions will not
> work.  Consider:
>
> As seen on Op<noframes>pression is contrary to the To</noframes>rah...
>
> Here, you actually have to understand the semantics of the tags;
> just stripping them out will fail.

In a different context (people trying to get deceptive search engine
keyword) I've thought about this problem.  An example I (and I'm sure
others) have imagined is something like this

 ...
 <style type="text/css">
  .special  {display none; }
  ...
 </style>

 ...

 <h1 class=special>Britney Spears, Saddam Hussain, Matrix Reloaded<h1>
 <h1>Low cost mortgages</h1>

with the idea of getting one's mortgage scam come up with popular
search terms.

Now the only solution I can see to this kind of thing would be to actually
try to render the HTML using something like the guts of mozilla, producing
postscript from that and then using some ps to text translation and
examining that text.

That seems enormously expensive, and certainly not worth it for spam
filtering.

> I believe fighting HTML tricks will become a losing battle, just like
> keeping up with virus signatures.  Just as a lot of people simply
> ban all .exe files, I believe in the long run, we'll have to just ban
> HTML mail (which will be one of the few happy consequences of spam. :-))

I concur.

But on the whole, I believe that content filtering for fighting spam is
worse than futile.  But that is a flamefest upon which nobody has anything
new to say.

I will say one thing for content filtering.  I use it on a modest site to
send everything with an SA score >= 5 to a the dsbl.org open relay tester.
(see
  http://dsbl.org/faq-help
for details on how to help)

About 1 in 5 that are checked generate a new listing on dsbl.org.

The relevant bit in mimedefang-filter for this is

  if ($hits >= 5) {
       add_recipient('spamtrap at slauson.com');
  }

in filter_end after the call to spam_assassin_check.

Spam trap is just a pipe to /usr/local/spamtrap, which comes with the DSBL
tester suite.  (You MUST configure it to not test those IPs that you can
legitimately relay through).

-j

-- 
Jeffrey Goldberg                            http://www.goldmark.org/jeff/
 Relativism is the triumph of authority over truth, convention over justice
 Hate spam?  Boycott MCI! http://www.goldmark.org/jeff/anti-spam/mci/