[Mimedefang] utf-8 issue?

Wed Jan 29 08:54:40 EST 2020

Hi all.

I ask you very much to continue the thread started in 2017. The problem
persists, and it is really annoying.

In brief: Mimedefang dies while message checking if *all *of following
conditions happen:

1. Attachment extension is "bad" and the attachment should be quarantined.

2. Attachment file name is non-ASCII and so why is encoded in UTF-8.

3. UTF-8 header encoding in original message is "incorrect" in some way.
Not sure what it means; many other Unicode filenames are processed and
quarantined properly.

I have supplied example strings to the maillist in 2018, and can add
more examples. Fresh one:

Jan 29 13:13:29 mimedefang.pl[24376]: ACE0720490:
MDLOG,ACE0720490,bad_filename,%42D%426%41F.rar,application/x-rar,<XXXX at mail.ru>,<XXXX at MY.com>,=?UTF-8?B?0K3QptCf?=
Jan 29 13:13:29 mimedefang-multiplexor[27228]: ACE0720490: Slave 1
stderr: open body: Invalid argument at /usr/share/perl5/MIME/Entity.pm
line 1892.
Jan 29 13:13:29 mimedefang-multiplexor[27228]: Slave 1 died prematurely
-- check your filter rules
Jan 29 13:13:29 postfix/cleanup[3830]: ACE0720490: milter-reject:
END-OF-MESSAGE from f558.i.mail.ru[94.100.184.166]: 4.7.1 Service
unavailable - try again later;

Any ideas or workarounds? Changing mime-tools version changes line
number in Entity.pm, and nothing more.

Older examples are below.

Alexey

On 25.06.2018 17:17, Alexey Lobanov wrote:

> Hi all.
>
> I continue the years old thread related to improper processing of
> Unicode strings either in Mimedefang or Perl libs. Last message
> https://lists.roaringpenguin.com/pipermail/mimedefang/2017-December/038247.html
>
> As far as I can see, the exception happens only when Mimedefang tries
> to remove an attachment with non-ASCII (e.g, Russian) name. If
> attachment is permitted and passed, exception does not appear. Looks
> like also, that other Russian names in same message do not generate an
> exception; not sure in exception logging scheme. Please note also that
> about 50% of attachments in our environment have Cyrillic names in
> UTF-8, but the issue happens few times a year only.
>
> The example log ("use diagnostics;" in mimedefang.pl) is below.
>
> The visible names of attachments are: "Ат. профессора.rar", "ВАК
> дмн.rar", "Диплом БДМУ 1982.rar" and "Категория 2016.rar".
>
> The MIME headers for first attachment are:
>
> Content-Type: application/octet-stream; 
> 	name="=?UTF-8?B?0JDRgi4g0L/RgNC+0YTQtdGB0YHQvtGA0LAucmFy?="
> Content-Disposition: attachment; filename="=?UTF-8?B?0JDRgi4g0L/RgNC+0YTQtdGB0YHQvtGA0LA=?=
> 	=?UTF-8?B?LnJhcg==?="
> Content-Transfer-Encoding: base64
> X-Attachment-Id: f_jimx17ff8
>
> Debian Jessie, Mimedefang 2.83 built from source, no embedded Perl
> mode, Mime-Tools 5.009, Perl v5.20.2.
>
> Yes, I have the whole message and I can provide any other
> non-confidential details from it.
>
> ====
>
> Jun 20 18:26:52 mail mimedefang.pl[25718]: C6CE82122B:
> MDLOG,C6CE82122B,bad_filename,%410%442.
> %43F%440%43E%444%435%441%441%43E%440%430.rar,application/octet-stream,<x at gmail.com>,<x at my.com>,Re:
> LEC
> Jun 20 18:26:52 mail mimedefang-multiplexor[25717]: C6CE82122B: Worker
> 0 stderr: Strings with code points over 0xFF may not be mapped into
> in-memory file
> Jun 20 18:26:52 mail mimedefang-multiplexor[25717]: C6CE82122B: Worker
> 0 stderr:        handles (#1)
> Jun 20 18:26:52 mail mimedefang-multiplexor[25717]: C6CE82122B: Worker
> 0 stderr:     (W utf8) You tried to open a reference to a scalar for
> read or append
> Jun 20 18:26:52 mail mimedefang-multiplexor[25717]: C6CE82122B: Worker
> 0 stderr:     where the scalar contained code points over 0xFF. 
> In-memory files
> Jun 20 18:26:52 mail mimedefang-multiplexor[25717]: C6CE82122B: Worker
> 0 stderr:     model on-disk files and can only contain bytes.
> Jun 20 18:26:52 mail mimedefang-multiplexor[25717]: C6CE82122B: Worker
> 0 stderr:
> Jun 20 18:26:52 mail mimedefang.pl[25718]: C6CE82122B:
> MDLOG,C6CE82122B,bad_filename,%412%410%41A
> %434%43C%43D.rar,application/octet-stream,<x at gmail.com>,<x at my.com>,Re: LEC
> Jun 20 18:26:52 mail mimedefang.pl[25718]: C6CE82122B:
> MDLOG,C6CE82122B,bad_filename,%414%438%43F%43B%43E%43C
> %411%414%41C%423
> 1982.rar,application/octet-stream,<x at gmail.com>,<x at my.com>,Re: LEC
> Jun 20 18:26:52 mail mimedefang.pl[25718]: C6CE82122B:
> MDLOG,C6CE82122B,bad_filename,%41A%430%442%435%433%43E%440%438%44F
> 2016.rar,application/octet-stream,<x at gmail.com>,<x at my.com>,Re: LEC
> Jun 20 18:26:52 mail mimedefang.pl[25718]: C6CE82122B:
> MDLOG,C6CE82122B,mail_in,,,<x at gmail.com>,<x at my.com>,Re: LEC
> Jun 20 18:26:52 mail mimedefang-multiplexor[25717]: C6CE82122B: Worker
> 0 stderr: Uncaught exception from user code:
> Jun 20 18:26:52 mail mimedefang-multiplexor[25717]: C6CE82122B: Worker
> 0 stderr:        open body: Invalid argument at
> /usr/share/perl5/MIME/Entity.pm line 1892.
> Jun 20 18:26:52 mail mimedefang-multiplexor[25717]: C6CE82122B: Worker
> 0 stderr:       
> MIME::Entity::print_bodyhandle(MIME::Entity=HASH(0xa10a63c),
> GLOB(0x9b07fe8)) called at /usr/share/perl5/MIME/Entity.pm line 1873
> Jun 20 18:26:52 mail mimedefang-multiplexor[25717]: C6CE82122B: Worker
> 0 stderr:       
> MIME::Entity::print_body(MIME::Entity=HASH(0xa10a63c),
> GLOB(0x9b07fe8)) called at /usr/share/perl5/MIME/Entity.pm line 1785
> Jun 20 18:26:52 mail mimedefang-multiplexor[25717]: C6CE82122B: Worker
> 0 stderr:        MIME::Entity::print(MIME::Entity=HASH(0xa10a63c),
> GLOB(0x9b07fe8)) called at /usr/share/perl5/MIME/Entity.pm line 1846
> Jun 20 18:26:52 mail mimedefang-multiplexor[25717]: C6CE82122B: Worker
> 0 stderr:        MIME::Entity::print_body
> Jun 20 18:26:52 mail mimedefang-multiplexor[25717]: C6CE82122B: Worker
> 0 stderr: (MIME::Entity=HASH(0xa106420), GLOB(0x9b07fe8)) called at
> /usr/local/bin/mimedefang.pl line 6160
> Jun 20 18:26:52 mail mimedefang-multiplexor[25717]: C6CE82122B: Worker
> 0 stderr:       
> main::do_scan("/var/spool/MIMEDefang/mdefang-W13qP1p") called at
> /usr/local/bin/mimedefang.pl line 5663
> Jun 20 18:26:52 mail mimedefang-multiplexor[25717]: C6CE82122B: Worker
> 0 stderr:        main::handle_scan("C6CE82122B",
> "/var/spool/MIMEDefang/mdefang-W13qP1p") called at
> /usr/local/bin/mimedefang.pl line 5600
> Jun 20 18:26:52 mail mimedefang-multiplexor[25717]: C6CE82122B: Worker
> 0 stderr:        main::do_main_loop() called at
> /usr/local/bin/mimedefang.pl line 5426
> Jun 20 18:26:52 mail mimedefang-multiplexor[25717]: C6CE82122B: Worker
> 0 stderr:        main::main called at /usr/local/bin/mimedefang.pl
> line 7521
> Jun 20 18:26:52 mail mimedefang-multiplexor[25717]: Worker 0 died
> prematurely -- check your filter rules
>
> ========