in reply to Re^3: deal with incorrectly set utf8 flag
in thread deal with incorrectly set utf8 flag

Thank you for your wisdom.
  • Comment on Re^4: deal with incorrectly set utf8 flag

Replies are listed 'Best First'.
Re^5: deal with incorrectly set utf8 flag
by ikegami (Patriarch) on Mar 27, 2009 at 16:53 UTC
    Actually, looks like utf8::encode is just as dumb as _utf8_off.
    $ perl -MDevel::Peek -MEncode=_utf8_on,_utf8_off,encode -e' _utf8_on( my $x = "\200\201" ); # Invalid UTF-8 utf8::encode( my $utf8 = $x ); my $enc = encode("UTF-8", $x); _utf8_off( my $off = $x ); Dump $x; Dump $utf8; Dump $enc; Dump $off; ' PV = 0x8165280 "\200\201"\0 [UTF8 "\x{1}@"] PV = 0x81623f0 "\200\201"\0 PV = 0x81920f8 "\357\277\275\357\277\275"\0 PV = 0x81ff040 "\200\201"\0

    Except when the flag isn't on.

    $ perl -MDevel::Peek -MEncode=_utf8_on,_utf8_off,encode -e' my $x = "\200\201"; utf8::encode( my $utf8 = $x ); my $enc = encode("UTF-8", $x); _utf8_off( my $off = $x ); Dump $x; Dump $utf8; Dump $enc; Dump $off; ' PV = 0x8165278 "\200\201"\0 PV = 0x816b608 "\302\200\302\201"\0 PV = 0x8195050 "\302\200\302\201"\0 PV = 0x81ff038 "\200\201"\0

    Your pick.