[Mimedefang] Spamassassin and MimeDefang custom rules
Paul Murphy
Paul.Murphy at argentadiscovery.com
Wed Oct 15 05:30:04 EDT 2008
Jon,
>/etc/mail/sa-mimedefang.cf
>In this config file I have the following custom test
>header BAD_TEST Subject =~ /xyzzy/i
>score BAD_TEST 6.00
>describe BAD_TEST Local Rule
This file is NOT parsed for rulesets - only for the main SA configuration options.
>Is there some way to capture a log of SA activity so I can perhaps see
>where I am going wrong?
Change your MD configuration to leave the /var/spool/MIMEDefang/mdefang-<msgid> directory in place (start with -d option), restart MD, send a test message, then take out the -d, find the directory (grep the mal log to get the message ID), become your filter user, and then in that directory, run:
spamassassin -p /etc/mail/sa-mimedefang.cf -t -d < INPUTMSG
This will run SA with your settings, and in debug mode - it won't produce exactly the same scores as MD due to several issues, but it will show whether your local rules are being considered, and whether they are being matched.
The output will be something like this: (I've added line numbers to assist in the commentary)
1 [27281] dbg: logger: adding facilities: all
2 [27281] dbg: logger: logging level is DBG
3 [27281] dbg: generic: SpamAssassin version 3.1.7
4 [27281] dbg: config: score set 0 chosen.
5 [27281] dbg: util: running in taint mode? yes
6 [27281] dbg: util: taint mode: deleting unsafe environment variables, resetting PATH
7 [27281] dbg: util: PATH included '/opt/kde3/bin', keeping
8 [27281] dbg: util: PATH included '/opt/gnome/bin', keeping
9 [27281] dbg: util: PATH included '/usr/games', keeping
10 [27281] dbg: util: PATH included '/home/defang/bin', keeping
11 [27281] dbg: util: PATH included '/usr/bin/X11', keeping
12 [27281] dbg: util: PATH included '/bin', keeping
13 [27281] dbg: util: PATH included '/usr/bin', keeping
14 [27281] dbg: util: PATH included '/usr/local/bin', keeping
15 [27281] dbg: util: PATH included '/packages/bin', which doesn't exist, dropping
16 [27281] dbg: util: PATH included '/usr/sbin', keeping
17 [27281] dbg: util: PATH included '/sbin', keeping
18 [27281] dbg: util: PATH included '.', which is not absolute, dropping
19 [27281] dbg: util: final PATH set to: /opt/kde3/bin:/opt/gnome/bin:/usr/games:/home/defang/bin:/usr/bin/X11:/bin:/usr/bin:/usr/local/bin:/usr/sbin:/sbin
20 [27281] dbg: message: ---- MIME PARSER START ----
21 [27281] dbg: message: main message type: text/plain
22 [27281] dbg: message: parsing normal part
23 [27281] dbg: message: added part, type: text/plain
24 [27281] dbg: message: ---- MIME PARSER END ----
25 [27281] dbg: dns: is Net::DNS::Resolver available? yes
26 [27281] dbg: dns: Net::DNS version: 0.46
27 [27281] dbg: config: using "/etc/mail/spamassassin" for site rules pre files
28 [27281] dbg: config: read file /etc/mail/spamassassin/init.pre
29 [27281] dbg: config: read file /etc/mail/spamassassin/v310.pre
30 [27281] dbg: config: read file /etc/mail/spamassassin/v312.pre
31 [27281] dbg: config: using "/var/lib/spamassassin/3.001007" for sys rules pre files
32 [27281] dbg: config: read file /var/lib/spamassassin/3.001007/updates_spamassassin_org.pre
33 [27281] dbg: config: using "/var/lib/spamassassin/3.001007" for default rules dir
34 [27281] dbg: config: read file /var/lib/spamassassin/3.001007/updates_spamassassin_org.cf
35 [27281] dbg: config: using "/etc/mail/spamassassin" for site rules dir
36 [27281] dbg: config: read file /etc/mail/spamassassin/10_report_template.cf
37 [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_adult.cf
38 [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_bayes_poison_nxm.cf
39 [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_evilnum0.cf
40 [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_evilnum1.cf
41 [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_evilnum2.cf
42 [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_genlsubj.cf
43 [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_header.cf
44 [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_html.cf
45 [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_obfu.cf
46 [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_oem.cf
47 [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_random.cf
48 [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_spoof.cf
49 [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_stocks.cf
50 [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_unsub.cf
51 [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_uri0.cf
52 [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_whitelist.cf
53 [27281] dbg: config: read file /etc/mail/spamassassin/72_sare_bml_post25x.cf
54 [27281] dbg: config: read file /etc/mail/spamassassin/72_sare_redirect_post3.0.0.cf
55 [27281] dbg: config: read file /etc/mail/spamassassin/99_local.cf
56 [27281] dbg: config: read file /etc/mail/spamassassin/99_sare_fraud_post25x.cf
57 [27281] dbg: config: read file /etc/mail/spamassassin/FuzzyOcr.cf
58 [27281] dbg: config: read file /etc/mail/spamassassin/ImageInfo.cf
59 [27281] dbg: config: read file /etc/mail/spamassassin/bogus-virus-warnings.cf
60 [27281] dbg: config: read file /etc/mail/spamassassin/local.cf
61 [27281] dbg: config: read file /etc/mail/spamassassin/sa-mimedefang.cf
62 [27281] dbg: config: read file /etc/mail/spamassassin/tripwire.cf
63 [27281] dbg: config: using "/home/defang/.spamassassin" for user state dir
64 [27281] dbg: config: using "/etc/mail/sa-mimedefang.cf" for user prefs file
65 [27281] dbg: config: read file /etc/mail/sa-mimedefang.cf
...
Update files processed:
96 [27281] dbg: config: using "/var/lib/spamassassin/3.001007/updates_spamassassin_org/empty.pre" for included file
97 [27281] dbg: plugin: fixed relative path: /var/lib/spamassassin/3.001007/updates_spamassassin_org/25_uribl.cf
98 [27281] dbg: config: using "/var/lib/spamassassin/3.001007/updates_spamassassin_org/25_uribl.cf" for included file
99 [27281] dbg: config: read file /var/lib/spamassassin/3.001007/updates_spamassassin_org/25_uribl.cf
100 [27281] dbg: plugin: fixed relative path: /var/lib/spamassassin/3.001007/updates_spamassassin_org/20_anti_ratware.cf
101 [27281] dbg: config: using "/var/lib/spamassassin/3.001007/updates_spamassassin_org/20_anti_ratware.cf" for included file
102 [27281] dbg: config: read file /var/lib/spamassassin/3.001007/updates_spamassassin_org/20_anti_ratware.cf
103 [27281] dbg: plugin: fixed relative path: /var/lib/spamassassin/3.001007/updates_spamassassin_org/25_razor2.cf
...
Blacklist checks:
318 [27281] dbg: uridnsbl: domains to query: untiesticks.com questiontrains.com
319 [27281] dbg: dns: checking RBL sa-other.bondedsender.org., set bsp-untrusted
320 [27281] dbg: dns: checking RBL combined.njabl.org., set njabl-lastexternal
321 [27281] dbg: dns: checking RBL combined.njabl.org., set njabl
...
Header and body regular expression checks:
333 [27281] dbg: check: running tests for priority: 0
334 [27281] dbg: rules: running header regexp tests; score so far=0
335 [27281] dbg: rules: ran header rule __HAS_MSGID ======> got hit: "<"
336 [27281] dbg: rules: ran header rule __SANE_MSGID ======> got hit: "<MUjhdgbjdmcacWS at UntieSticks.com>
337 [27281] dbg: rules: "
338 [27281] dbg: rules: ran header rule __MSGID_OK_HOST ======> got hit: "@UntieSticks.com>"
339 [27281] dbg: rules: ran header rule __SUBJECT_ENCODED_QP ======> got hit: "=?ISO-8859-1?Q?"
340 [27281] dbg: rules: ran header rule __SARE_HEAD_MIME_VALID ======> got hit: "1.0"
341 [27281] dbg: rules: ran header rule __CT ======> got hit: "m"
342 [27281] dbg: rules: ran header rule __TOCC_EXISTS ======> got hit: "<"
343 [27281] dbg: rules: ran header rule __HAS_SUBJECT ======> got hit: "Y"
344 [27281] dbg: rules: ran header rule __SARE_WHITELIST_FLAG ======> got hit: "R"
345 [27281] dbg: rules: ran header rule __CTYPE_MULTIPART_ALT ======> got hit: "multipart/alternative"
346 [27281] dbg: rules: ran header rule __CTYPE_HAS_BOUNDARY ======> got hit: "boundary"
347 [27281] dbg: rules: ran header rule __MIME_VERSION ======> got hit: "1"
348 [27281] dbg: rules: ran header rule BAD_ENC_HEADER ======> got hit: "=?ISO-8859-1?Q?You could get up to 750 GBP =2D Instantly with a 1-minute "
...
Summarising SA tests:
485 [27281] dbg: rules: running header regexp tests; score so far=32.795
486 [27281] dbg: rules: running body-text per-line regexp tests; score so far=32.795
487 [27281] dbg: uri: running uri tests; score so far=32.795
488 [27281] dbg: rules: running raw-body-text per-line regexp tests; score so far=32.795
489 [27281] dbg: rules: running full-text regexp tests; score so far=32.795
490 [27281] dbg: check: running tests for priority: 900
491 [27281] dbg: rules: running meta tests; score so far=32.795
492 [27281] dbg: rules: running header regexp tests; score so far=32.795
493 [27281] dbg: rules: running body-text per-line regexp tests; score so far=32.795
494 [27281] dbg: uri: running uri tests; score so far=32.795
495 [27281] dbg: rules: running raw-body-text per-line regexp tests; score so far=32.795
496 [27281] dbg: rules: running full-text regexp tests; score so far=32.795
497 [27281] dbg: check: running tests for priority: 1000
498 [27281] dbg: rules: running meta tests; score so far=32.795
499 [27281] dbg: rules: running header regexp tests; score so far=32.795
...
AWL update:
512 [27281] dbg: plugin: Mail::SpamAssassin::Plugin::AutoLearnThreshold=HASH(0x9514bb0) implements 'autolearn_discriminator'
513 [27281] dbg: learn: auto-learn: currently using scoreset 3, recomputing score based on scoreset 1
514 [27281] dbg: learn: auto-learn: message score: 32.795, computed score for autolearn: 25.953
515 [27281] dbg: learn: auto-learn? ham=1, spam=7, body-points=23.698, head-points=6.255, learned-points=0
516 [27281] dbg: learn: auto-learn? yes, spam (25.953 > 7)
517 [27281] dbg: learn: initializing learner
518 [27281] dbg: learn: learning spam
...
Bayes learn:
556 [27281] dbg: bayes: database connection established
557 [27281] dbg: bayes: found bayes db version 3
558 [27281] dbg: bayes: Using userid: 2
559 [27281] dbg: bayes: 90e28644e964898dcc6015cc5f5f318aa876d4a1 at sa_generated already learnt correctly, not learning twice
560 [27281] dbg: learn: initializing learner
...
And finally, a decision:
561 [27281] dbg: check: is spam? score=32.795 required=5
562 [27281] dbg: check: tests=BAD_ENC_HEADER,EXCUSE_REMOVE,HTML_MESSAGE,NO_RECEIVED,NO_RELAYS,RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E4_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,SUBJECT_EXCESS_QP,URIBL_BLACK,URIBL_OB_SURBL,URI_NOVOWEL
563 [27281] dbg: check: subtests=__CT,__CTYPE_HAS_BOUNDARY,__CTYPE_MULTIPART_ALT,__HAS_MSGID,__HAS_SUBJECT,__HTML_LINK_IMAGE,__LOCAL_PP_NONPPURL,__MIME_HTML,__MIME_QP,__MIME_VERSION,__MSGID_OK_HOST,__NONEMPTY_BODY,__SANE_MSGID,__SARE_BLACK_FG_COLOR,__SARE_BODY_BLNK_5_100,__SARE_HAS_BG_COLOR,__SARE_HAS_FG_COLOR,__SARE_HEAD_MIME_VALID,__SARE_HTML_HAS_A,__SARE_HTML_HAS_BR,__SARE_HTML_HAS_FONT,__SARE_HTML_HAS_IMG,__SARE_HTML_HAS_P,__SARE_HTML_HAS_TITLE,__SARE_META_MURTY3,__SARE_URI_ANY,__SARE_WHITELIST_FLAG,__SARE_WHITE_BG_COLOR,__SARE_WHITE_FG_COLOR,__SUBJECT_ENCODED_QP,__TAG_EXISTS_BODY,__TAG_EXISTS_CENTER,__TAG_EXISTS_HEAD,__TAG_EXISTS_HTML,__TAG_EXISTS_META,__TOCC_EXISTS,__TVD_BODY,__TVD_MIME_ATT_TP,__UNUSABLE_MSGID
The key part is on line 64 - sa-mimedefang.cf is read as a user preferences file, so it is not parsed for rulesets.
Add your rules to /etc/spamassassin/local.cf instead.
Best Wishes,
Paul.
_______________________________________________________________________
Argenta Discovery Ltd, 8-9 Spire Green Centre, Harlow, Essex, CM19 5TR
Registered in England No. 3671653
_______________________________________________________________________
More information about the MIMEDefang
mailing list