[Mimedefang] replace_with_url with multiple small attachments

[Matthew S. Cramer]

On Wed, Dec 20, 2006 at 09:07:14PM +1100, Rolf wrote:

> I cannot quite get my head around what scripting I need to deal with  
> multiple attachments where no single one is larger than the permitted  
> maximum but where the total number of attachments does exceed that  
> size.  Where one or more parts are greater than the max permitted  
> then those parts are replaced with links as expected.
> 
> As the size test is against $entity then it is only triggered if that  
> part is greater that the chosen maximum.  I tried triggering max size  
> against the size of "./INPUTMSG"  but it seems not as simple as that.
> 
> Appreciate some pointers to scripting so that I could reduce the  
> delivered mail size to below the max. size incorporating as many  
> replace_with_url() replacements as necessary to construct such a  
> message.

We had that same problem - we replace any attachment > 7MB because our
internal mail system rejects mails > 10MB.  We used 7MB in MIMEDefang
because when we check attachment size with this code:

    # determine size of an attachment
    if (defined($body)) {
        $size = (stat($entity->bodyhandle->path))[7];
    }

in filter the size does not include MIME encoding, but our internal
mail system only knows raw size.

That was working well but we kept getting occasional reports of mails
still being rejected by the internal system for size even though they
passed the gateways running MIMEDefang.  Since our users never saved
the mails when they called the Helpdesk we basically assumed they were
wrong and never researched their complaints extensively.

Then one night at the bar it dawned on me (thank you, Scotch) that the
way my code worked if a mail contained 10 1MB attachments it would
pass the check for size on the gateways but be bounced by the internal
system.  A quick dd if=/dev/urandom of=this.file bs=1MB count=1 and an
email with this.file attached 10 times proved that our check was
broken for that scenario.  Looking back at the users' complaints, they
were getting emails with 3 or 4 office documents, each document 3 or 4
MB. 

So I added this code to my mimedefang-filter in filter:

    if (-s "./INPUTMSG" > 10000*1024) {
        unless (lc($type) =~ "text/[plain,html]") {

            $date = $time{'dd Month yyyy H AM tz', time};
            # convert to GMT
            $pdate = ParseDate($date);
            $pdate = Date_ConvTZ($pdate,"EST","GMT");
            $secs = &UnixDate($pdate,"%s");
            $secs = $time{'dd Month yyyy H AM', $secs};

            my $salt = random_salt();

            if (action_replace_with_url_arm($entity, "SANITIZED",
                "SANITIZED", "SANITIZED", "SANITIZED", $salt)) {
               return 1;
            } else {
               return action_tempfail("Your mail contains an
                attachment which I am currently unable to process.
                Please try to send again later.");
            }
        }
    }

which runs after my old 7MB check (too lazy to test to confirm that
adding the latter eliminated the need for the former).  It takes any
mail which is 10MB and replaces any attachment with a url as long as
that attachment it not text.  If someone sent an email with a total
size of text attachments > 10MB this would still be broken, but that
is so unlikely I don't care.  Such an email would deserve to be
bounced.  :)

Since filter is called for each attachment, there is no need to build
in iteration beyond that which MIMEDefang naturally performs (David,
you rock!).  The code above will keep getting executed until the
entire mail is < 10MB since filte ris called once for each attachment.

I also recommend setting a maximum size in sendmail to prevent DoS,
a size you won't likely encounter but if were attacked in such a
manner, could tolerate until the attacker was neutralized.

BTW, I use my own hacked version of action_replace_with_url and do
some funky stuff involving time stamps because any Microsoft
Executable gets quarantined for 24 hours before the link in the email
actually works.  We do this with a combination of NFS mounts, perl
scripts, and a PHP site interface to bypass the quarantine period for
that developer who absolutely MUST HAVE that zip file with the new
.dll file for his app right then.  But our Helpdesk is required to
tell the user she had better be damn sure this isn't a virus before
they "publish" the attachment from the quarantine to the download
area ahead of the 24 hours period.

For anyone who gets infected with Viruses via email a good 12
hours before any antivirus vendor has a signature update, and whose
Business would not tolerate completely blocking MS Executables (such
as was the case with us), this approach will effectively remove email
as a threat vector with little to no impact on users.


HTH,

Matt

-- 
Matthew S. Cramer <mscramer at armstrong.com>          Office: 717-396-5032
Project Manager, Planning and Service Management    Fax:    717-396-5590
Armstrong World Industries, Inc.                    Cell:   717-917-7099