Re^3: deal with incorrectly set utf8 flag

this is documented as an internal function that shouldn't be relied on.

It means you should normally use utf8::encode or Encode::encode 'UTF-8'.

$ perl -MDevel::Peek -MEncode=_utf8_on,_utf8_off,encode -e'
    _utf8_on( $x = "\342\231\240" );

    utf8::encode( my $utf8 = $x );
    my $enc = encode("UTF-8", $x);
    _utf8_off( my $off = $x );

    Dump $x;
    Dump $utf8;
    Dump $enc;
    Dump $off;
'
  PV = 0x8165280 "\342\231\240"\0 [UTF8 "\x{2660}"]
  PV = 0x81623f0 "\342\231\240"\0
  PV = 0x81920e8 "\342\231\240"\0
  PV = 0x81ff040 "\342\231\240"\0
[download]

But if _utf8_on or equivalent was wrongly used, _utf8_off is appropriate.

Comment on Re^3: deal with incorrectly set utf8 flag Select or Download Code

Replies are listed 'Best First'.
Re^4: deal with incorrectly set utf8 flag by Anonymous Monk on Mar 27, 2009 at 16:48 UTC
Thank you for your wisdom.	[reply]
Re^5: deal with incorrectly set utf8 flag by ikegami (Patriarch) on Mar 27, 2009 at 16:53 UTC
Actually, looks like `utf8::encode` is just as dumb as `_utf8_off`. `$ perl -MDevel::Peek -MEncode=_utf8_on,_utf8_off,encode -e' _utf8_on( my $x = "\200\201" ); # Invalid UTF-8 utf8::encode( my $utf8 = $x ); my $enc = encode("UTF-8", $x); _utf8_off( my $off = $x ); Dump $x; Dump $utf8; Dump $enc; Dump $off; ' PV = 0x8165280 "\200\201"\0 [UTF8 "\x{1}@"] PV = 0x81623f0 "\200\201"\0 PV = 0x81920f8 "\357\277\275\357\277\275"\0 PV = 0x81ff040 "\200\201"\0` [download] Except when the flag isn't on. `$ perl -MDevel::Peek -MEncode=_utf8_on,_utf8_off,encode -e' my $x = "\200\201"; utf8::encode( my $utf8 = $x ); my $enc = encode("UTF-8", $x); _utf8_off( my $off = $x ); Dump $x; Dump $utf8; Dump $enc; Dump $off; ' PV = 0x8165278 "\200\201"\0 PV = 0x816b608 "\302\200\302\201"\0 PV = 0x8195050 "\302\200\302\201"\0 PV = 0x81ff038 "\200\201"\0` [download] Your pick.	[reply] [d/l] [select]