[Mimedefang] MIME::Entity not handling Charset => 'utf-8' correctly?

Wed Feb 20 19:55:23 EST 2013

Should probably mention I'm running:

perl-MIME-Types-1.28-2.el6.noarch
perl-MIME-Lite-3.027-2.el6.noarch
perl-MIME-tools-5.427-4.el6.noarch
perl-MIME-Base32-1.02a-1.el6.noarch
mimedefang-2.73-3.el6.x86_64
perl-5.10.1-127.el6.x86_64

on CentOS 6.3.

On Feb 20, 2013, at 5:20 PM, Philip Prindeville <philipp_subx at redfish-solutions.com> wrote:

> Hi.
> 
> I'm trying to generate a message as a footer in mimedefang-filter (in filter_end()) when I see certain message contents, but I'm running into what looks like a bug.  I've reproduced it here:
> 
> [philipp]$ cat test.pl
> #!/usr/bin/perl -w
> 
> use strict;
> use warnings;
> 
> use MIME::Entity;
> use MIME::QuotedPrint;
> use HTML::Entities;
> 
> my $string = decode_qp("Ellipsis=E2=80=A6\n");
> 
> utf8::upgrade($string);
> 
> print "string: ", $string;
> 
> print "hex: ", unpack('H*', $string), "\n";
> 
> my $msg = encode_entities($string, '"<>&');
> 
> my @strings = (
> "<html>\n",
> $msg,
> "</html>\n"
> );
> 
> 
> my $html = MIME::Entity->build(
>  Top => 0,
>  Type => 'text/html',
>  Encoding => 'quoted-printable',
>  Charset => 'utf-8',
>  Data => [ @strings ],
> );
> 
> print $html->as_string(), "\n";
> 
> exit 0;
> [philipp]$ ./test.pl
> string: Ellipsis…
> hex: 456c6c6970736973e280a60a
> Content-Type: text/html; charset="utf-8"
> Content-Disposition: inline
> Content-Transfer-Encoding: quoted-printable
> 
> <html>
> Ellipsis=C3=A2=C2=80=C2=A6
> </html>
> 
> [philipp]$ 
> 
> 
> from what I can tell, if I do a Data::Dumper() on $html->bodyhandle()->{'MBS_Data'} then it looks like the 3 UTF characters (0xa280a8) have been converted into \x{a2}, \x{80}, \x{a8} instead…  Which I don't understand, since I've explicitly called the Charset out as being 'utf-8'. It looks like the string is being interpreted as latin1, not utf8.
> 
> What am I doing wrong?
> 
> Thanks,
> 
> -Philip
> 
> _______________________________________________
> NOTE: If there is a disclaimer or other legal boilerplate in the above
> message, it is NULL AND VOID.  You may ignore it.
> 
> Visit http://www.mimedefang.org and http://www.roaringpenguin.com
> MIMEDefang mailing list MIMEDefang at lists.roaringpenguin.com
> http://lists.roaringpenguin.com/mailman/listinfo/mimedefang