http://qs1969.pair.com?node_id=1090613


in reply to Re^2: Default encoding rules leave me puzzled... (use open qw/ :std :locale /;
in thread Default encoding rules leave me puzzled...

So, does Perl assume by default, even in a UTF-8 environment, that it should output everything in Latin-1 ?

Perl tries to not convert anything at all, automatically.

And since Latin-1 (mostly?) maps the first 256 codepoints 1:1 to bytes, outputting something without any conversion is the same as outputting it as Latin-1.

Note that this round-trips binary data, which means that if your scripts or input use UTF-8, and you don't use utf8;, the output will be UTF-8 again.

But, Latin-1 is limited to codepoints up to 255, so if something higher than that shows up in your string, perl falls back to UTF-8 (and warns).

(As always, I'm linking to Encodings and Unicode in Perl, in the hope that it's useful to you).

Perl 6 - the future is here, just unevenly distributed
  • Comment on Re^3: Default encoding rules leave me puzzled... (use open qw/ :std :locale /;
  • Download Code

Replies are listed 'Best First'.
Re^4: Default encoding rules leave me puzzled... (use open qw/ :std :locale /;
by kzwix (Sexton) on Jun 20, 2014 at 13:08 UTC
    I'm sorry, but I think that your "Perl tries to not convert anything at all, automatically" statement is wrong.
    I mean, else, why would a string internally stored as UTF-8 be converted to Latin-1 when sent to the standard output ?
    (That is, without having used any funny encoding/decoding/layer stuff...)
[reply]

      It's your use utf8; statement that explicitly converts the source code to perl's internal format, which happens to be latin-1 in this case. The output operation then doesn't convert anything. Maybe I should have said that no implicit conversion takes place, because I don't coutn the use utf8; as automatic/implicit.

      Perl 6 - the future is here, just unevenly distributed
[reply]
[d/l]
[select]

      I mean, else, why would a string internally stored as UTF-8 be converted to Latin-1 when sent to the standard output ?

      Because you are printing the string, not its internal representation. The layout of a scalar is irrelevant.

[reply]

In Section Seekers of Perl Wisdom