[Mimedefang] two md_check_against_smtp_server questions

[Yizhar Hurwitz]

HI.

 > From: John Rudd <john at rudd.cc>
 > 2) Has anyone set up a means of caching results?  I don't want to hit my
 > back-line servers constantly with these requests.  I would prefer to
 > have results cached for, say, 2 hours.  I'm trying to think of a good
 > way to do this.

You can take a look here:
Ray Ferguson's fancy version:
http://www.mimedefang.org/kwiki/index.cgi?RecipientCheckBDBCache

But his version is quite complex and difficult to follow (at least for me).
Also - there is an important bug in the above version,
look at the lines (from Ray :
    if ( $stat == "REJECT" ...
    } elsif ( $stat == "CONTINUE" ...
The above is wrong because he is using numeric "==" for string comprasion,
and will produce the wrong results.
I will send him (Ray) an email about this off the list also.


I am in a similar boat, and have just started playing with my own cached 
md_check_against_smtp version.

However my situation is different - I need it for a relativly low traffic 
system, between 100-200 recipients, and no more then 6 concurent MD slaves.
Therefor I think that I can afford spending RAM and hold the recipients cache in 
a simple perl hash in RAM of each slave, while I write updates to a file on disk.
I currently cache only positive results (CONTINUE, OK).

I have just started using something last week, and am still working on it.

I have not yet added timestamps to the cache and plan to add it later,
so the code below will cache positive response forever -
but this will be changed and fixed.

This is what I Currently have in mimedefang-filter:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

### Valid Recipients Cache:
my $vrc_file= '/home/defang/vrc-sdbm';
my %vrc_disk;
my %vrc_ram;

#***********************************************************************
# %PROCEDURE: filter_initialize
#***********************************************************************
sub filter_initialize {
  if ($CheckRecipientEnable) {
   if (-e $vrc_file) {
    ### Load Valid Recipient Cache from file to ram:
    tie (%vrc_disk, 'SDBM_File', $vrc_file, O_RDONLY, 0666) or die "Cannot 
create VRC file, $_";
    foreach (keys %vrc_disk) {
     $vrc_ram{$_} = 1;
    }
    untie (%vrc_disk);
   }
   else {
    ### Create a new empty disk cache:
    tie (%vrc_disk, 'SDBM_File', $vrc_file, O_RDWR|O_CREAT, 0666) or die "Cannot 
create VRC file, $_";
    untie (%vrc_disk);
   }
  }
}

#***********************************************************************
# %PROCEDURE: filter_recipient
#***********************************************************************
sub filter_recipient
{
  my($recip, $sender, $ip, $host, $first, $helo, $rcpt_mailer, $rcpt_host, 
$rcpt_addr) = @_;
  if ($CheckRecipientEnable) {
   if ($vrc_ram{$recip}) {
    return ('CONTINUE', 'OK');
   }
   else {
    my ($stat,$msg,$code) = md_check_against_smtp_server($sender, $recip, 
$HostName, $CheckRecipientServer);
    if ($stat eq 'CONTINUE') {
     $vrc_ram{$recip} = 1;
     tie (%vrc_disk, 'SDBM_File', $vrc_file, O_RDWR, 0666) or die "Cannot tie 
$vrc_file for write, $_";
     $vrc_disk{$recip} = 1;
     untie (%vrc_disk);
    }
    return ($stat, $msg, $code);
   }
  }
  return ('CONTINUE', 'OK');
}

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I would like to add my own questions to the list about the same issue:

How efficient is a simple perl hash when accessing it?
i.e. when I write:

   if ($vrc_ram{$recip}) .....

So assuming that I have 100 keys (recipients) in the hash,
Does perl need to go over all the 100 keys in the hash,
or does it do some magic tricks and find it more efficiently?

Which db file type is best for storing and accessing such a cache:
for example:
DB_File
NDBM_File
SDBM_File
GDBM_File
I really don't understand the actual differences between them,
although I've read their man pages (but didn't googled yet)...
Is a disk cache using "tie" with one of the above database formats,
more efficient then a regular perl hash in RAM?

I do preffer to use a simple database without additional database software like 
sql server which seems an overkill for my needs.
(I currently don't use db for other things in the filter).

Any comments are welcome...

Yizhar Hurwitz
http://yizhar.mvps.org