Re^3: Default encoding rules leave me puzzled... (use open qw/ :std :locale /;

So, does Perl assume by default, even in a UTF-8 environment, that it should output everything in Latin-1 ?

Perl tries to not convert anything at all, automatically.

And since Latin-1 (mostly?) maps the first 256 codepoints 1:1 to bytes, outputting something without any conversion is the same as outputting it as Latin-1.

Note that this round-trips binary data, which means that if your scripts or input use UTF-8, and you don't use utf8;, the output will be UTF-8 again.

But, Latin-1 is limited to codepoints up to 255, so if something higher than that shows up in your string, perl falls back to UTF-8 (and warns).

(As always, I'm linking to Encodings and Unicode in Perl, in the hope that it's useful to you).

Perl 6 - the future is here, just unevenly distributed

Comment on Re^3: Default encoding rules leave me puzzled... (use open qw/ :std :locale /; Download Code

Replies are listed 'Best First'.
Re^4: Default encoding rules leave me puzzled... (use open qw/ :std :locale /; by kzwix (Sexton) on Jun 20, 2014 at 13:08 UTC
I'm sorry, but I think that your "Perl tries to not convert anything at all, automatically" statement is wrong. I mean, else, why would a string internally stored as UTF-8 be converted to Latin-1 when sent to the standard output ? (That is, without having used any funny encoding/decoding/layer stuff...)	[reply]
Re^5: Default encoding rules leave me puzzled... (use open qw/ :std :locale /; by moritz (Cardinal) on Jun 20, 2014 at 17:27 UTC
It's your `use utf8;` statement that explicitly converts the source code to perl's internal format, which happens to be latin-1 in this case. The output operation then doesn't convert anything. Maybe I should have said that no implicit conversion takes place, because I don't coutn the `use utf8;` as automatic/implicit. Perl 6 - the future is here, just unevenly distributed	[reply] [d/l] [select]
Re^5: Default encoding rules leave me puzzled... (use open qw/ :std :locale /; by ikegami (Patriarch) on Jun 23, 2014 at 03:28 UTC
I mean, else, why would a string internally stored as UTF-8 be converted to Latin-1 when sent to the standard output ? Because you are printing the string, not its internal representation. The layout of a scalar is irrelevant.	[reply]


Keep It Simple, Stupid
	PerlMonks