[Mimedefang] Greylist DB addition fails silently?
Justin
justin at othius.com
Wed Jun 23 23:33:10 EDT 2004
On Thu, 24 Jun 2004, Roland Pope wrote:
> ----- Original Message -----
> From: "Justin" <justin at othius.com>
>> I have modified Steven Rocha's implementation
>> (http://lists.roaringpenguin.com/pipermail/mimedefang/2004-February/020126.html)
>> which I believe is a modification of Jonas' implementation. My modified
>> version uses a PostgreSQL database in place of Berkley DB and allows
>> you to specify action to take (white/black/grey) based on cidr/host
>> address, using subnet 0/0 as the default action.
>>
>> I will clean it up and post if there's interest.
>
> I would be very interested in a copy of this as I have wanted to use
> greylists, but needed to have a shared DB as I have multiple MX's.
Be careful with single point of failure. I believe many here have noted
that it's better to just have an independent greylist db on each relay.
The attached snippet of filter should get you going with greylisting and
postgresql. Note that it also includes some popauthdb code adapted from
Kevin McGrail's example.
And while I'm at it let me make sure I give credit to:
jonas - for the original work
steven - for the modified version i adapted
kevin - for popauthdb bit
puremagic - gl_triplets table layout (adapted from their mysql table)
david - mimedefang
You'll need a postgresql database setup with tables as defined
something like this (beware of occasional line wrap):
Table "public.gl_triplets"
Column | Type | Modifiers
----------------+-----------------------------+---------------------
id | integer | not null default
nextval('public.gl_triplets_id_seq'::text)
relay_ip | inet |
mail_from | character varying(255) |
rcpt_to | character varying(255) |
block_expires | timestamp without time zone | not null
record_expires | timestamp without time zone | not null
blocked_count | bigint | not null default 0
passed_count | bigint | not null default 0
aborted_count | bigint | not null default 0
create_time | timestamp without time zone | not null
last_update | timestamp without time zone | not null
Indexes:
"triplet_key" primary key, btree (id)
"ip_from_to" btree (relay_ip, mail_from, rcpt_to)
Table "public.md_subnet_rules"
Column | Type | Modifiers
--------+-----------------------+-----------
subnet | inet | not null
action | character varying(10) |
Indexes:
"md_subnet_rules_pkey" primary key, btree (subnet)
Some more columns might be helpful, such as created_date and a comment
area on md_subnet_rules, but these two are all the attached filter snippet
requires.
You should populate the md_subnet_rules table with something like the
following as a beginning:
subnet | action
-----------------+--------
10.1.1.0/24 | white
127.0.0.1 | white
0.0.0.0/0 | grey
The record at the end is very important, and is used to define the default
action you wish to take. If you wished, you could set the 0/0 record to
white, and then only greylist specific subnets, like comcast, apnic, etc.
It's up to you.
Drop a note if there are any problems with the code.
-Justin
-------------- next part --------------
#*********************************************************************
# Greylist
#Settings for greylisting.
#
# For an explanation of what the purpose of this is, and maybe a hint as to
# what values to enter, "check http://projects.puremagic.com/greylisting/".
# I think they recommend something like this:
# $gdb_black = 1*60*60;
# $gdb_grey = 5*60*60;
# $gdb_white = 36*24*60*60;
# $gdb_subnet = 1;
#
#
# If $greylist is 1, greylisting will be used.
#
# Greylisting is done on a triplet of sending hosts IP, mail from: and
# rcpt to:.
#
# When a session with a new triplet arrives, all sessions with that
# triplet will be tempfailed for $gdb_black seconds.
# After $gdb_black seconds it will be white-listed for $gdb_grey
# seconds.
# If a session for the triplet arrives within the $gdb_grey white-listing
# period, it will then be white-listed for $gdb_white seconds.
# If a session for a triplet arrives within the $gdb_white white-listing
# period, it will be white listed for another $gdb_white seconds.
#
# If $gdb_subnet is true, only the first 3 octes of the IP-addresses will be
# used in the greylist.
# If $gdb_from_domain is true, only the domain part of the mail from: address
# will be used in the greylist.
# If $gdb_to_domain is true, only the domain part of the rcpt to: address
# will be used in the greylist.
# If $gdb_from_strip is true, some stuff in the user part of the mail from:
# address will be replaced in order to handle mailinglists and some other
# stuff better.
# If $gdb_to_strip is true, some stuff in the user part of the rcpt to:
# address will be replaced in order to handle use parameters and some other
# stuff better.
#
#
# Make sure you set $gdb_dsn, $gdb_username and $gdb_password
# to be relevent for your site
#***********************************************************************
$greylist = 1;
$gdb_dsn = 'DBI:Pg:dbname=mimedefang';
$gdb_username = 'mimedefang';
$gdb_password = 'password';
$gdb_black = 15*60; #If you don't exist, you have to wait for
$gdb_grey = 24*60*60; #If you do exist, you have this long from when you showed up, to send us something
$gdb_white = 36*24*60*60; #If you sent us something in the window, you have this long before we forget you
$gdb_subnet = 0;
$gdb_from_domain = 0;
$gdb_from_strip = 1;
$gdb_to_domain = 0;
$gdb_to_strip = 1;
$gdb_log = 1;
use DBI;
use Date::Manip;
$gdb_connection;
###############################
#Greylist Subroutines ########
###############################
#Open database connection lazily
sub greylist_database_open() {
if(!defined($gdb_connection)) {
$gdb_connection = DBI->connect($gdb_dsn, $gdb_username, $gdb_password);
}
return defined($gdb_connection);
}
#Strip strings
sub address_strip ($) {
my($a) = @_;
$a = "" if (!defined($a));
$a =~ s/^[<\[]//;
$a =~ s/[>\]]$//;
return lc($a);
}
# return a time string...
sub time_string($) {
my ($time) = @_;
my $h = int($time / (60*60));
$time = $time % (60*60);
my $m = int($time / 60);
my $s = $time % 60;
my $r = "";
$r.="$h hours, " if ($h);
$r.="$m minutes and " if ($h || $m);
$r.="$s seconds";
return $r;
}
#Strip strings for use in the greylist.
sub greylist_strip ($) {
my($a) = @_;
$a =~ s/;/:/g;
return $a;
}
sub greylist_strip_mail($$$) {
my($a,$d,$s) = @_;
$a = address_strip($a);
my $au = $a;
my $ad = $a;
$ad =~ s/.*@([^@]*)$/$1/;
$au =~ s/@[^@]*$//;
if ($d) {
$au = "*";
} elsif ($s) {
$au =~ s/(.+)\+.*$/$1/;
my $aut;
my $autt = $au;
do {
$aut = $autt;
$autt =~ s/^(|.*[^a-z0-9])[a-f0-9]*\d[a-f0-9]*(|[^a-z0-9].*)$/$1#$2/;
} until ($autt eq $aut);
$au = $aut if ($aut =~ /[a-z0-9]/);
#$au =~ s/[^-a-z0-9_.#]/?/g;
}
return greylist_strip($au."@".$ad);
}
sub greylist_strip_ip($) {
my($a) = @_;
$a =~ s/(.*)\.[0-9]+$/$1\.*/ if (defined($gdb_subnet) && $gdb_subnet);
return greylist_strip(address_strip($a));
}
sub greylist_strip_triplet(@) {
my(@p) = @_;
my($i,$s,$r) = @p;
my $sr;
my $sn;
$s = greylist_strip_mail($s,(defined($gdb_from_domain) && $gdb_from_domain),(defined($gdb_from_strip) && $gdb_from_strip));
$r = greylist_strip_mail($r,(defined($gdb_to_domain) && $gdb_to_domain),(defined($gdb_to_strip) && $gdb_to_strip));
$i = greylist_strip_ip($i);
return ($i,$s,$r);
}
#Checks if a triplet is in the grey-list.
# Returns seconds until the triplet will be accepted, or -1 for error.
sub greylist_check($$$) {
my ($ip,$sender,$recipient) = greylist_strip_triplet(@_);
my $result = -1;
greylist_database_open();
my $now = scalar localtime();
if ($gdb_connection) {
my $event = "";
$sth = $gdb_connection->prepare("SELECT id, create_time,last_update,record_expires,block_expires,passed_count FROM gl_triplets WHERE relay_ip='$ip' AND mail_from=".$gdb_connection->quote($sender)." AND rcpt_to=".$gdb_connection->quote($recipient).";");
$sth->execute;
my ($rid,$created,$modified,$reset,$accepted,$count);
$sth->bind_columns( undef, \$rid, \$created, \$modified, \$reset, \$accepted, \$count );
if (!$sth->fetch()) {
my $sql = "INSERT INTO gl_triplets (relay_ip,mail_from,rcpt_to,block_expires,record_expires,create_time,last_update) values (inet '$ip',".$gdb_connection->quote($sender).",".$gdb_connection->quote($recipient).", timestamp '$now' + interval '$gdb_black seconds', timestamp '$now' + interval '$gdb_grey seconds', timestamp '$now', timestamp '$now');";
$gdb_connection->do( $sql );
$result = $gdb_black;
$event = 'new';
} else {
if (Date_Cmp($now, $accepted) < 0 ) { #$now <= $accepted
#At this point they are retrying under their blacklist window
$result = UnixDate($accepted,"%s")-UnixDate($now,"%s");
$event = 'black';
#Log a note that they retried under our window
$gdb_connection->do("UPDATE gl_triplets SET blocked_count=blocked_count+1, last_update=timestamp '$now' WHERE id=$rid");
} elsif (Date_Cmp($now, $reset) < 0) { #$now <= $reset
#At this point they are retrying past the blacklist window, but inside their expiration window.
$result = 0;
$event = 'white';
#Log a note that they retried sucessfully, and make sure they are updated to the whitelist window
$gdb_connection->do("UPDATE gl_triplets SET passed_count=passed_count+1,record_expires=timestamp '$now' + interval '$gdb_white seconds',last_update=timestamp '$now' WHERE id=$rid");
} else { #$now > $reset
#At this point they are retrying past their expiration
$gdb_connection->do("UPDATE gl_triplets SET blocked_count=blocked_count+1, block_expires=timestamp '$now' + interval '$gdb_black seconds', record_expires=timestamp '$now' + interval '$gdb_grey seconds', last_update=timestamp '$now' WHERE id=$rid");
$result = $gdb_black;
$event = 'old';
}
}
$sth->finish;
md_syslog('info', "greylist: $event; $result; $ip; $sender; $recipient") if (defined($gdb_log) && $gdb_log);
}
return $result;
}
sub popauthdb_check($$) {
my ($dbfile,$relay_ip) = @_;
my (%db);
tie (%db, "DB_File", $dbfile, O_RDONLY, 0, $DB_HASH) || print('critical', "Could not tie to database: $dbfile!");
my ($result) = defined($db{$relay_ip});
untie %db;
return $result;
}
#***********************************************************************
# PROCEDURE: filter_recipient
# %ARGUMENTS:
# recipient, sender, ip, host, first, helo, rcpt_mailer, rcpt_host, rcpt_addr
# %RETURNS:
# action
# %DESCRIPTION:
# Called just after RCPT TO
# Requires -t
#***********************************************************************
sub filter_recipient ($$$$$$$$$) {
my($recipient, $sender, $ip, $hostname, $first, $helo, $rcpt_mailer, $rcpt_host, $rcpt_addr) = @_;
my($subnet_action); #Database tells us what to do with given IP ranges
md_syslog('info', "filter_recipient: From $sender to $rcpt_addr at $rcpt_host with $rcpt_mailer");
#Open a connection to our database
greylist_database_open();
if($gdb_connection) {
$sth = $gdb_connection->prepare("SELECT action from md_subnet_rules where inet '$ip' <<= subnet order by subnet desc limit 1;");
$sth->execute;
$sth->bind_columns(undef, \$subnet_action);
if(!$sth->fetch()) {
md_syslog('warning', "filter_recipient: subnet rules check returned error!");
return ('TEMPFAIL', "Something is not working right here. Please try again.");
}
$sth->finish;
}
if(popauthdb_check($popauthdbfile,$ip)) {
$subnet_action = "white";
}
#Check greylist
if ($subnet_action eq "grey") {
$grey = greylist_check($ip,$sender,$recipient);
if ($grey > 0) {
my $greys = time_string($grey);
md_syslog('info', "MDLOG,$MsgID,grey,$grey,$ip,$sender,$recipient,?");
return ('TEMPFAIL', "We will accept the mail in $greys.");
} elsif ($grey < 0) {
md_syslog('warning', "filter_recipient: greylist_check returned error!");
return ('TEMPFAIL', "Something is not working right here. Please try again.");
}
}
return ('CONTINUE', "Ok, go ahead.");
}
sub filter_initialize {
#INITIALIZE THE MODULES AND PARAMETERS NEEDED TO SETUP A TIE TO THE POP BEFORE SMTP AUTH DATABASE
#Thanks to Ole Craig for comments on this functionality of this feature
#
# NOTE: the database MUST BE READABLE BY THE DEFANG USER (try "chgrp defang /etc/mail/popauth.db" or
# even chmod 744 /etc/mail/popauth.db)
use DB_File;
our ($popauthdbfile);
$popauthdbfile = "/etc/mail/popauth.db";
}
sub filter_cleanup {
#Close the database connection if it's open
$gdb_connection->disconnect() if{$gdb_connection};
}
More information about the MIMEDefang
mailing list