[Mimedefang] Filtering based on X-Mailer or X-MIMEOLE header?

Paul Murphy Paul.Murphy at argentadiscovery.com
Fri Nov 24 07:51:39 EST 2006


Hi,

In the ever-escalating war, I'm having problems with some spammers
sending stock scams with large chunks of random text either side, and
while I'm updating my SA rules daily, I never seem able to keep ahead of
the game with these.   Eventually DCC and Razor2 catch up, but the first
couple of hours is always a problem and I end up cobbling together my
own rules to block specific spams.

I've been considering alternative approaches, and one which seems
attractive on the surface is to further analyse the message headers for
indications of spammyness - we've already got the Received headers, the
sender and recipient, and the Subject covered, but can the others be
used to provide an indication that the content is spam?  SA already
considers some of these in deciding things like whether the message
claims to be sent using Outlook but doesn't have the correct headers to
support this claim, but this is highly specific and not generally very
helpful.

I decided to look at the X-Mailer and X-MIMEOLE headers specifically,
and to extract these in a fairly ad-hoc way for each message, and then
add the details of the message, SA score, and mailer to a database
table.

I'd then propose to adjust the SA score based on an analysis of the
history data.  The adjustment would be equal to 10% of (the mean score
minus one standard deviation), so a small offset in most cases. 
Obviously my policy is up to me, so no-one has to do this, but I thought
I'd share my thoughts and experiences.

Applying this across a day's worth of traffic here produces some
promising results:

select count(*) as cnt,
        round(sum(score),2) as total,
        round(avg(score),2) as mean,
        round(min(score),2) as min,
        round(stddev(score),2) as stddev,
        round((avg(score)-stddev(score))/10,2) as adj,
        left(mailer,50) as mailer
from mail_msg
where mailer is not null
group by left(mailer,50)
order by mean;

+-----+---------+--------+---------+--------+-------+----------------------------------------------------+
| cnt | total   | mean   | min     | stddev | adj   | mailer           
                                 |
+-----+---------+--------+---------+--------+-------+----------------------------------------------------+
|   1 |  -11.43 | -11.43 |  -11.43 |   0.00 | -1.14 | StrongMail
Enterprise 3.1.5(2.00.223)              |
|  22 | -226.31 | -10.29 | -101.21 |  28.81 | -3.91 | Microsoft CDO for
Windows 2000                     |
|   4 |  -16.95 |  -4.24 |   -5.23 |   0.80 | -0.50 | Kana Connect 6   
                                 |
|   2 |   -5.65 |  -2.82 |   -2.97 |   0.15 | -0.30 | Roving Constant
Contact 0 (http//www.constantconta |
|   1 |   -2.60 |  -2.60 |   -2.60 |   0.00 | -0.26 | Microsoft
Outlook, Build 10.0.6626                 |
|   1 |   -2.60 |  -2.60 |   -2.60 |   0.00 | -0.26 | Microsoft Outlook
IMO, Build 9.0.6604 (9.0.2911.0) |
|   4 |  -10.05 |  -2.51 |   -2.58 |   0.04 | -0.25 | Novell GroupWise
Internet Agent 6.5.4              |
|   1 |   -2.51 |  -2.51 |   -2.51 |   0.00 | -0.25 | Lotus Notes
Release 6.5.1 January 21, 2004         |
|   1 |   -2.47 |  -2.47 |   -2.47 |   0.00 | -0.25 | BBC EBS Custom
Mailer v2                           |
|   1 |   -2.46 |  -2.46 |   -2.46 |   0.00 | -0.25 | Microsoft
Outlook, Build 10.0.4024                 |
|   2 |   -4.76 |  -2.38 |   -2.41 |   0.03 | -0.24 | Lotus Notes
Release 6.5.4 March 27, 2005           |
|   1 |   -2.35 |  -2.35 |   -2.35 |   0.00 | -0.23 | Apple Mail
(2.750)                                 |
|   1 |   -2.33 |  -2.33 |   -2.33 |   0.00 | -0.23 | GlobalCrossing   
                                 |
|   1 |   -2.30 |  -2.30 |   -2.30 |   0.00 | -0.23 | Internet Mail
Service (5.5.2658.27)                |
|   4 |   -8.95 |  -2.24 |   -2.34 |   0.08 | -0.23 | Internet Mail
Service (5.5.2653.19)                |
|   6 |  -13.27 |  -2.21 |   -2.53 |   0.44 | -0.26 | Microsoft Office
Outlook 11                        |
|   7 |  -14.66 |  -2.09 |   -2.50 |   0.50 | -0.26 | Microsoft
Exchange V6.0.6603.0                     |
|  10 |  -19.32 |  -1.93 |   -2.22 |   0.12 | -0.21 | Internet Mail
Service (5.5.2658.3)                 |
|   2 |   -3.85 |  -1.92 |   -1.93 |   0.01 | -0.19 | Microsoft MimeOLE
V6.00.3790.504                   |
|   2 |   -3.50 |  -1.75 |   -2.60 |   0.85 | -0.26 | Lotus Notes
Release 6.5.3 September 14, 2004       |
|  10 |  -17.15 |  -1.72 |   -2.09 |   0.22 | -0.19 | ColdFusion MX
Application Server                   |
|   6 |  -10.25 |  -1.71 |   -2.30 |   0.77 | -0.25 | Microsoft
Exchange V6.5.6944.0                     |
|   1 |   -1.67 |  -1.67 |   -1.67 |   0.00 | -0.17 |
pyroclasticmailsplatterer 0.0.1                    |
|   1 |   -1.64 |  -1.64 |   -1.64 |   0.00 | -0.16 | AOL Email 22250  
                                 |
|   1 |   -1.64 |  -1.64 |   -1.64 |   0.00 | -0.16 | Microsoft MimeOLE
V6.00.2800.1807                  |
|   1 |   -1.62 |  -1.62 |   -1.62 |   0.00 | -0.16 | Lotus Notes
653HF860 June 07, 2006                 |
|  17 |  -26.96 |  -1.59 |   -1.63 |   0.03 | -0.16 | Mach5 Mailer-3.00
PID{0fe5a8aa-6c14-4bc2-8eae-07ae |
|   7 |  -10.92 |  -1.56 |   -2.03 |   0.50 | -0.21 | HTML Mime mail
class                               |
|  12 |  -18.39 |  -1.53 |   -2.47 |   0.89 | -0.24 | Microsoft
Exchange V6.5.7226.0                     |
|   1 |   -1.51 |  -1.51 |   -1.51 |   0.00 | -0.15 | Microsoft MimeOLE
V6.00.2800.1441                  |
|  71 | -105.50 |  -1.49 |   -2.60 |   0.99 | -0.25 | Microsoft
Exchange V6.5                            |
|   1 |   -1.46 |  -1.46 |   -1.46 |   0.00 | -0.15 | TRX CORREX
4.9.0.1 (MIME-tools 5.411) on UK1448    |
|   3 |   -4.22 |  -1.41 |   -1.50 |   0.13 | -0.15 | aspNetEmail ver
2.5.0.0                            |
|   4 |   -5.64 |  -1.41 |   -2.46 |   1.05 | -0.25 | Microsoft
Exchange V6.0.6249.0                     |
|  15 |  -20.55 |  -1.37 |   -1.46 |   0.05 | -0.14 | Maestro
MailServer                                 |
|   1 |   -1.35 |  -1.35 |   -1.35 |   0.00 | -0.14 | QUALCOMM Windows
Eudora Version 7.0.1.0            |
|   3 |   -4.02 |  -1.34 |   -1.48 |   0.14 | -0.15 | 9.0 SE for
Windows sub 5017                        |
|   4 |   -5.33 |  -1.33 |   -1.81 |   0.48 | -0.18 | Microsoft MimeOLE
V6.00.3790.2663                  |
|   5 |   -6.53 |  -1.31 |   -1.92 |   1.22 | -0.25 | 3.5.14 build 759 
                                 |
|   1 |   -1.31 |  -1.31 |   -1.31 |   0.00 | -0.13 | Connaught email
server v4.30                       |
|   1 |   -1.28 |  -1.28 |   -1.28 |   0.00 | -0.13 | TFS Secure
Messaging Server /470900753/470901753/4 |
|   1 |   -1.26 |  -1.26 |   -1.26 |   0.00 | -0.13 | Lotus Notes
Release 5.0.12   February 13, 2003     |
|   1 |   -1.24 |  -1.24 |   -1.24 |   0.00 | -0.12 | Lotus Notes
Release 6.5.1 January 28, 2004         |
|   1 |   -1.05 |  -1.05 |   -1.05 |   0.00 | -0.10 | MessageFocus
Launcher (v1.5-live)                  |
|  64 |  -66.02 |  -1.03 |  -13.98 |   2.38 | -0.34 | Unknown          
                                 |
|   2 |   -1.89 |  -0.95 |   -0.97 |   0.02 | -0.10 | Microsoft MimeOLE
V6.00.3790.2757                  |
|   1 |   -0.81 |  -0.81 |   -0.81 |   0.00 | -0.08 | Delano e-Business
Interaction Server, Send E-mail  |
|   4 |   -2.50 |  -0.63 |   -2.60 |   1.14 | -0.18 | Internet Mail
Service (5.5.2657.72)                |
|  18 |   -9.88 |  -0.55 |   -2.53 |   1.94 | -0.25 | Microsoft Outlook
Express 6.00.2900.2869           |
|   1 |   -0.50 |  -0.50 |   -0.50 |   0.00 | -0.05 | Lotus Domino Web
Server Release 7.0 August 18, 200 |
|   3 |   -0.95 |  -0.32 |   -2.60 |   1.84 | -0.22 | PHPMailer
[version 1.73]                           |
|   6 |   -0.57 |  -0.10 |   -0.70 |   0.52 | -0.06 | EDMAIL R6.00.02  
                                 |
| 235 |   -2.20 |  -0.01 |   -2.20 |   0.14 | -0.02 | Novell GroupWise
Internet Agent 7.0.1              |
|   5 |    0.00 |   0.00 |    0.00 |   0.00 |  0.00 | Lotus Notes
Release 6.5.2 June 01, 2004            |
|   1 |    0.00 |   0.00 |    0.00 |   0.00 |  0.00 | Groupwise 7.0    
                                 |
|   1 |    0.00 |   0.00 |    0.00 |   0.00 |  0.00 | Willco Mailer    
                                 |
|   1 |    0.00 |   0.00 |    0.00 |   0.00 |  0.00 | Microsoft
Outlook, Build 10.0.2616                 |
|   9 |    0.00 |   0.00 |    0.00 |   0.00 |  0.00 | Groupwise 7.0.1  
                                 |
|   1 |    0.32 |   0.32 |    0.32 |   0.00 |  0.03 | Microsoft CDO for
Exchange 2000                    |
|   7 |    4.12 |   0.59 |    0.59 |   0.00 |  0.06 | Mach5 Mailer-3.00
PID{f49a49e2-0754-48c4-946b-18ba |
|  14 |    8.62 |   0.62 |    0.00 |   0.54 |  0.01 | Lotus Notes
Release 6.5.5  CCH1 March 07, 2006     |
|   1 |    0.66 |   0.66 |    0.66 |   0.00 |  0.07 | MIMELite 3.01
(F2.72; A1.62; B3.01; Q3.01)         |
|   8 |    5.91 |   0.74 |    0.00 |   0.49 |  0.02 | Lotus Notes
Release 6.0 September 26, 2002         |
|   1 |    0.82 |   0.82 |    0.82 |   0.00 |  0.08 | RMSmtp           
                                 |
|   1 |    0.89 |   0.89 |    0.89 |   0.00 |  0.09 | AspQMail 2.0 4.11
(QSM2E82E6F)                     |
|  16 |   19.84 |   1.24 |    1.08 |   0.08 |  0.12 | Mach5 Mailer-3.00
PID{72ff768d-3829-4713-b8b1-1bd0 |
|   2 |    3.22 |   1.61 |    1.60 |   0.00 |  0.16 | Silverpop Mailer
2.0                               |
|   1 |    2.17 |   2.17 |    2.17 |   0.00 |  0.22 | ExclamationSoft
Corporation Mail Version 2.0       |
|   2 |    4.49 |   2.24 |    2.15 |   0.09 |  0.22 | Dundas Mailer
Control 1.0                          |
|   2 |    4.53 |   2.27 |    2.22 |   0.05 |  0.22 | ExclamationSoft
Corporation Mail Version 3.0       |
|   1 |    2.36 |   2.36 |    2.36 |   0.00 |  0.24 | YAB Mailer
yabd(7,main)                            |
|   2 |    5.86 |   2.93 |    2.37 |   0.56 |  0.24 | AtMail 4.03      
                                 |
|   1 |    3.04 |   3.04 |    3.04 |   0.00 |  0.30 | EMS              
                                 |
|   1 |    3.36 |   3.36 |    3.36 |   0.00 |  0.34 | american golf
Mail Shots Distribution Server       |
|   6 |   22.08 |   3.68 |   -1.10 |   2.69 |  0.10 | eBizmailer3.6    
                                 |
|   1 |    4.03 |   4.03 |    4.03 |   0.00 |  0.40 | mailenstein
E-Mail-Marketing System www.mailenstei |
|   2 |    8.13 |   4.06 |    4.03 |   0.04 |  0.40 | Lotus Notes
Release 6.0.2CF1 June 9, 2003          |
|  10 |   40.83 |   4.08 |    2.89 |   0.45 |  0.36 | Highwire Mailer  
                                 |
|   8 |   46.75 |   5.84 |   -2.60 |  12.85 | -0.70 | Microsoft Outlook
IMO, Build 9.0.2416 (9.0.2910.0) |
|   2 |   12.04 |   6.02 |    5.45 |   0.57 |  0.55 | eshotonline (http
                                 |
|   1 |    7.19 |   7.19 |    7.19 |   0.00 |  0.72 | Openwave
WebEngine, version 2.8.16.1 (webedge20-10 |
|   1 |    8.41 |   8.41 |    8.41 |   0.00 |  0.84 | MIME             
                                 |
|   3 |   26.91 |   8.97 |    1.40 |   7.57 |  0.14 | Microsoft Outlook
Express 6.00.2800.1437           |
|   3 |   34.27 |  11.42 |    4.26 |   6.56 |  0.49 | Microsoft Outlook
Express 6.00.2800.1506           |
|   3 |   41.60 |  13.87 |   10.78 |   4.35 |  0.95 | Microsoft Outlook
Express 6.00.2800.1807           |
|   1 |   14.60 |  14.60 |   14.60 |   0.00 |  1.46 | Microsoft Outlook
Express 5.50.4807.1700           |
|  24 |  392.98 |  16.37 |   -2.60 |  16.97 | -0.06 | Microsoft Office
Outlook, Build 11.0.5510          |
|  10 |  172.61 |  17.26 |    7.26 |   8.33 |  0.89 | Microsoft Outlook
Express 6.00.2800.1106           |
|   1 |   20.05 |  20.05 |   20.05 |   0.00 |  2.00 | vlyfqmtzxso
ifgfil - 6.0                           |
|   1 |   20.31 |  20.31 |   20.31 |   0.00 |  2.03 | Microsoft Outlook
Express 6.00.2800.1409           |
|   1 |   23.25 |  23.25 |   23.25 |   0.00 |  2.33 |
SquirrelMail/1.4.3a                                |
|   1 |   24.38 |  24.38 |   24.38 |   0.00 |  2.44 | Microsoft
Outlook, Build 10.0.3416                 |
|   1 |   24.91 |  24.91 |   24.91 |   0.00 |  2.49 | The Bat!
(v2.00.3) Business                        |
|   4 |  108.28 |  27.07 |   15.20 |  16.39 |  1.07 | Microsoft Outlook
Express 6.00.2800.1158           |
|   9 |  251.21 |  27.91 |   -2.60 |  17.04 |  1.09 | Microsoft Office
Outlook, Build 11.0.6353          |
|   1 |   28.91 |  28.91 |   28.91 |   0.00 |  2.89 | The Bat!
(v2.12.00) UNREG / CD5BF9353B3B7091       |
|   1 |   30.01 |  30.01 |   30.01 |   0.00 |  3.00 | The Bat! (v2.11)
Business                          |
|   1 |   30.27 |  30.27 |   30.27 |   0.00 |  3.03 | The Bat! (v3.01
RC8) Educational                   |
|   1 |   30.92 |  30.92 |   30.92 |   0.00 |  3.09 | The Bat!
(v3.81.14 Beta) UNREG / CD5BF9353B3B7091  |
|   1 |   30.99 |  30.99 |   30.99 |   0.00 |  3.10 | The Bat!
(v3.0.2.2 Rush) UNREG / 77YIB4V52SDZ8OWAN |
|   1 |   31.02 |  31.02 |   31.02 |   0.00 |  3.10 | The Bat! (v2.0
Beta/1) Educational                 |
|   2 |   63.41 |  31.71 |   29.68 |   2.02 |  2.97 | The Bat! (v3.01
RC8) Professional                  |
|   1 |   31.91 |  31.91 |   31.91 |   0.00 |  3.19 | The Bat!
(v2.10.01) Personal                       |
|   1 |   32.13 |  32.13 |   32.13 |   0.00 |  3.21 | The Bat!
(v2.00.5) Business                        |
|   1 |   32.35 |  32.35 |   32.35 |   0.00 |  3.23 | The Bat! (v3.0)
Home                               |
|  34 | 1124.03 |  33.06 |    3.77 |  15.70 |  1.74 | Microsoft Outlook
Express 6.00.2900.2180           |
|   1 |   33.77 |  33.77 |   33.77 |   0.00 |  3.38 | The Bat!
(v3.0.1.33) Home                          |
|   1 |   33.82 |  33.82 |   33.82 |   0.00 |  3.38 | Microsoft Outlook
Express 5.00.2919.6700           |
|   1 |   34.15 |  34.15 |   34.15 |   0.00 |  3.41 | Microsoft Outlook
Express 4.72.3338.1              |
|   1 |   35.06 |  35.06 |   35.06 |   0.00 |  3.51 | The Bat!
(v3.5.25) UNREG / CD5BF9353B3B7091        |
|   1 |   35.09 |  35.09 |   35.09 |   0.00 |  3.51 | The Bat!
(v2.00.7) Educational                     |
|   1 |   35.11 |  35.11 |   35.11 |   0.00 |  3.51 | The Bat!
(v3.62.14) Home                           |
|   1 |   36.82 |  36.82 |   36.82 |   0.00 |  3.68 | The Bat!
(v2.10.01) Business                       |
|   1 |   37.26 |  37.26 |   37.26 |   0.00 |  3.73 | The Bat!
(v2.00.6) Personal                        |
|   1 |   37.69 |  37.69 |   37.69 |   0.00 |  3.77 | The Bat!
(v3.80.03) UNREG / CD5BF9353B3B7091       |
|   1 |   37.81 |  37.81 |   37.81 |   0.00 |  3.78 | The Bat!
(v3.0.0.8) Educational                    |
|   1 |   37.97 |  37.97 |   37.97 |   0.00 |  3.80 | The Bat!
(v3.0.0.15) Educational                   |
|   1 |   38.47 |  38.47 |   38.47 |   0.00 |  3.85 | The Bat!
(v2.00.2) Business                        |
|   1 |   38.63 |  38.63 |   38.63 |   0.00 |  3.86 | The Bat! (v3.0.1
RC7) Home                         |
|   1 |   39.40 |  39.40 |   39.40 |   0.00 |  3.94 | The Bat!
(v2.10.03) UNREG / CD5BF9353B3B7091       |
|   2 |   79.04 |  39.52 |   37.06 |   2.46 |  3.71 | Spam mashine - 7 
                                 |
|   1 |   40.10 |  40.10 |   40.10 |   0.00 |  4.01 | The Bat!
(v2.00.7) CD5BF9353B3B7091                |
|   1 |   40.69 |  40.69 |   40.69 |   0.00 |  4.07 | Microsoft Outlook
Express 5.50.4927.1200           |
|   1 |   42.01 |  42.01 |   42.01 |   0.00 |  4.20 | Microsoft
Internet Mail 4.70.1132                  |
|   1 |   42.86 |  42.86 |   42.86 |   0.00 |  4.29 | The Bat!
(v3.51.10) Educational                    |
|   1 |   43.57 |  43.57 |   43.57 |   0.00 |  4.36 | The Bat!
(v2.11.02) CD5BF9353B3B7091               |
|   1 |   47.65 |  47.65 |   47.65 |   0.00 |  4.76 | The Bat!
(v2.04.7) UNREG / CD5BF9353B3B7091        |
|   1 |   48.74 |  48.74 |   48.74 |   0.00 |  4.87 | Microsoft
Outlook, Build 10.0.2627                 |
|   1 |   51.21 |  51.21 |   51.21 |   0.00 |  5.12 | Microsoft Outlook
IMO, Build 9.0.2416 (9.0.2911.0) |
|  19 | 1036.58 |  54.56 |   51.03 |   2.90 |  5.17 | The Bat!
(v2.00.6) Business                        |
|   1 |   60.00 |  60.00 |   60.00 |   0.00 |  6.00 | The Bat! (v3.5)
Professional                       |
+-----+---------+--------+---------+--------+-------+----------------------------------------------------+
131 rows in set (0.10 sec)

Now, obviously I don't want to apply an adjustment based on a single
message, so it looks like I'll only apply the adjustment if we have 10
or more messages with that mailer included.

Some conclusions:

1.  Surprisingly, one of the most trustworthy mailers is from Microsoft
- but CDO is Collaboration Data Objects, which suggests that you have
spent a large amount of money on an Exchange Enterprise edition system,
which is unlikely for spammers.

2.  Everything I've seen which claims to be "The Bat!" is highly
suspect, despite the claims on their web site that its all forged - if
this is the case, they don't seem to have any real customers.

3.  Outlook Express 6.00.2900.2180 is primarily used by spammers - this
is probably because it is the most widely installed, being the unpatched
version from IE6.0 for Windows XP SP2, but it also shows a high average
score, and in fact out of 34 messages seen with this mailer, only 4
scored less than 10 in SA.  Three of these were stock scams, and the
fourth was an unsolicited CV from someone in India.  The current
suggested adjustment, of 1.74, would have caused all of the stock scams
to be blocked.

4.  Some spammers are very honest - 2 of them are sending using a
mailer called "Spam mashine"

5.  Some people are hopelessly out of date on what I presume are their
internal systems - the unpatched Lotus Notes 6.0 from 2002 is in use by
a major UK pharmaceutical company, and this sort of attitude probably
explains why their mail system shuts down for a week every time a new
virus outbreak happens.

Questions:

A.  Has anyone else ever done anything like this?  Any experiences to
share?
B.  The X-Mailer and X-MIMEOLE headers are not universal, and are under
the control of the sender in any case, so am I wasting my time with
this?  Or will it likely work for a little while, and then every spam
will have a unique mailer, or no header at all?
C.  Are there any other flaws in my idea?

Thanks, and congratulations if you read this far....

Best Wishes,

Paul.
-- 

-------------------------------------------------------
Paul Murphy
Head of I.T.
Argenta Discovery
Tel. 01279 645 554
Fax. 01279 645 646




More information about the MIMEDefang mailing list