[Mimedefang] Mailing lists, ham, and broken MUA's

[Steffen Kaiser]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mon, 3 May 2010, Philip Prindeville wrote:

> The problem is this: the message will be intelligible to English language 
> readers, but it will generate a lot of false positives for mailing list 
> recipients who aren't expecting to get non-English messages (or English 
> messages encoded in anything other than USASCII, ISO-8895-1, or UTF-8).

:-)

> If the message body is Content-Type: text/plain; charset=xxxx should it be 
> squashed down in the case of mailing list traffic for English language 
> mailing lists?

Nice idea. To make it really work, you should exempt the signature.
Meaning, there are people using their native spelling as
name.

> use Encode::First qw(encode_first);
>
> my $encodings = join('ascii', 'latin1', 'utf-8', $oldcharset);
my $encodings = join(',', 'ascii', 'latin1', 'utf-8');

"utf8" matches always, IMHO, but first you have to decode() the content,
which BTW I found problematic in its own, that's why I'm using
a "decode_first"-like function:

try decode with supplied charset, then check if it is good utf8,
then decode as latin1, which matches always.

That's the same with your sequence: $oldcharset will never reached
because you can always encode to 'utf-8'.

> my ($newcharset, $newlen) = encode_first($encodings, $string);
>
> if ($newlen<= length($string)) {
>    # use $newstr instead
> }

This check does not fit, IMHO: If you have a real, 7bit clean ASCII
message, it should be the same in any other multi-byte or 8bit
encodings, because they use ASCII as bases, don't they?

Your goal is to hide the Asian charset for English messages,
therefore I would use:

my %goodCharset = ( qw/ascii latin1 iso-8856-1/ );
if(!$goodCharset{lc $oldcharset} && $goodCharset{lc $newcharset}) {
 	# replace body
}
UTF-8 does not do any good, but hides the Asian font :-)

Regards,

- -- 
Steffen Kaiser
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iQEVAwUBS+AmW0gddVksjRnHAQJftAf/a2SNUOSJgVn8vGn19WTvNO4EwdCvOkSG
zHEVFr0oegd+/URgGcS/Sl4lZhIx2vuXXLZQu4bnMTFHcpZiJLTqd1xQupMGsgP6
V3TVNQ2mzUdz755Eh+QAxuMgAtxvlMzQSobOe89X8VfVr1pEukau02HXYZfUPNs1
zA77B0FGQfcCcPK4VwiHNT2nImgCySvVTLRR9LFM+sx1Jz7/aw0TwJb7Youm1ejd
+hfi5F6PosQHSVxz0CALSg2Do6zvQR64+KFrpBvXdgXar9g28eVjnu3disdp6w49
Z1x9tg2Fc4PskY7/uthbeH05OGmj89hPzVSbQbkTwuXNYyt50+CB4w==
=XADR
-----END PGP SIGNATURE-----