[Mimedefang] how much spam does the average large company get ?

Wed Nov 19 16:25:23 EST 2003

Matt,

Your best source of data, as it applies specifically to your servers, is
your maillog.  It will give you "hard numbers" for your management.

A while back, I posted a script, along with a sample of the output, that I
run nightly to parse "hard numbers" from my maillog.  If you look back
through the archives, you should find it.  It was about a month ago.

Below are some of the types of numbers you can pull.  These examples are
just SOME of the numbers from this morning's report.  But these should give
you all the numbers you need.  (And these dont include similar statistics
pulled from the Exchange servers, which run Trend Micro's content filter to
catch even more junk.

The numbers in these reports are used for exactly the same reasons you note.
They get plugged into graphs to plot trends and show spikes.  They let us
easily spot anomolies.  They justify hardware upgrade budget requests.

First, the output of "mailstats".  The "totals" will give you traffic volume
baselines.  In this case, we have sendmail set up to give the breakdowns in
a per-domain format, for each of our internal domains (munged here to simply
be an unrevealing DOMAIN1, DOMAIN2, etc.)

 M   msgsfr  bytes_from   msgsto    bytes_to  msgsrej msgsdis  Mailer
 4    26896     871843K    21119     850573K     1870    3933  esmtp
 9    12580     528863K    20289     709468K        9       2  DOMAIN1
11      163      12376K      162      11773K        1       0  DOMAIN2
12       30        357K       58        230K        0       0  DOMAIN3
13       13       8393K     1070      14104K        0       0  DOMAIN4
14      472      25758K      760      44953K        0       0  DOMAIN5
17     5590     119368K      639      13369K        3       0  DOMAIN6
18       10        933K        1         97K        0       0  DOMAIN7
19     6485     118742K     1236       6694K       54       3  local
=============================================================
 T    52239    1686633K    45334    1651261K     1937    3938
 C    53612                46760                 8179

DISTINCT HOSTS WITH WHOM CONNECTIONS WERE ATTEMPTED IN PRECEDING 24HRS:
    5762 Total distinct individual hosts

OUT OF ALL THOSE HOSTS WITH WHOM WE'VE ATTEMPTED A CONNECTION,
THE LATEST CONNECTION ATTEMPT FOR EACH BREAKS DOWN AS FOLLOWS:
    5310 Successfully completed transactions with us
      31 Reset the connection on us, during a transaction
     284 Timed-out on us, during a transaction
     131 Refused connections from us
       6 Failed for other reasons

SENDMAIL RULESET REJECTION TALLIES
      22 CheckFrom
      27 CheckMessageId
    1318 CheckSubject
     187 CheckTo
     473 check_mail
      50 check_rcpt
    2311 check_relay
    4388 total

MIMEDEFANG "MILTER" REJECTION TALLIES
     111 bad_filename
       2 charset_korean
       1 non_multipart
    2606 sa_discard_score
    4204 sa_quarantine_score
      11 suspicious_chars
      43 virus
    6978 total

SPAMASSASSIN SCORES
Msgs Scoring 5 or Higher: 11485
  Highest Recorded Score: 54.753
      Average Spam Score: 15.14

MIMEDEFANG STOPPED THE FOLLOWING VIRUSES
       2 Hits: W32/Yaha.E
       2 Hits: Worm/Bugbear.B
       1 Hits: Worm/Dumaru.A
       4 Hits: Worm/Gibe.C.1
      22 Hits: Worm/Klez.E
       3 Hits: Worm/MiMail.A1
       6 Hits: Worm/Yaha.M
       3 Hits: Worm/Yaha.P

MIMEDEFANG "MILTER" PROCESSING TIME TALLIES
Shortest time to process: 1ms
 Longest time to process: 54209ms
 Average time to process: 1903ms

AVERAGE MESSAGE SIZE: 34 KB

SMALLEST MESSAGE(S)
	Size: 1 Bytes
	Msg=hAIFOAs6019350

LARGEST MESSAGE(S)
	Size: 29764740 Bytes
	Msg=hAIM1ot7006121

AVERAGE RATE - MESSAGES PER MINUTE
  MIDNIGHT-8AM: 36
       8AM-5PM: 112
  5PM-MIDNIGHT: 54
       24 HOUR: 70

TOP 10 BUSIEST MINUTES:
  296 Msgs/Min @ 17:50
  281 Msgs/Min @ 13:54
  264 Msgs/Min @ 14:00
  264 Msgs/Min @ 13:59
  264 Msgs/Min @ 13:53
  263 Msgs/Min @ 08:48
  262 Msgs/Min @ 08:15
  255 Msgs/Min @ 08:43
  243 Msgs/Min @ 13:45

-----Original Message-----
From: Matt Cramer [mailto:mscramer at armstrong.com]
Sent: Wednesday, November 19, 2003 4:04 PM
To: mimedefang at lists.roaringpenguin.com
Subject: [Mimedefang] how much spam does the average large company get?

My management has asked me to make some graphs (blech) showing how good of
a job we do fighting spam.  I know what we do here - basically by
eliminating spams with large SpamAss scores, and doing much of the
sophisticated smtp filtering mimedefang allows, I've gotten our delivered
but flagged spam level down to about 20%.  Before I started, we were at
40%.  That is a good trend that I can show, but I am trying to get an idea
of what other companies see in terms of spam.  We are Fortune 500ish, with
about 7000 mail users.

I know the commercial spam fighting tools' marketing usually claims that
40-60% of a large company's mail is spam.  Is that accurate in others
experience?  Not that I don't trust marketing people, but they might have
a tendency to inflate that.  Our CEO doesn't tolerate bullshit numbers
when you present to him, so I'd feel better at defending my claim of
40-60% for others with maybe an impartial survey or even some anecdotes
from listmembers.  My research online found plenty of claims, but all of
it coming from research from anti-spam vendors.

Comments or suggestions welcome.

Matt

-- 
Matthew S. Cramer <mscramer at armstrong.com>          Office: 717-396-5032
Infrastructure Security Analyst                     Fax:    717-396-5590
Armstrong World Industries, Inc.                    Cell:   717-917-7099

_______________________________________________
MIMEDefang mailing list
MIMEDefang at lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang