kzwix has asked for the wisdom of the Perl Monks concerning the following question:
Hello, ô wise ones,
I'm using Perl under an UTF-8 Linux environment (LC_ALL='en_US.UTF-8'), with a perl script which is in an UTF-8-encoded file.
This script states "use utf8;" on one of its first lines, and then, I have some scalar definitions which use accentuated characters (I'm French, so there are some 'é', 'è', 'à', 'ô', 'ù', etc.)
However, when I print strings defined in the script, I see they are garbled (the accentuated characters come out as ugly white squares on the black background of my Putty terminal - Which is ALSO configured to use UTF-8)
Now, if I write :
binmode STDOUT, ':encoding(UTF-8)';
and do the same for STDERR, I get no more garbled characters, they all come out cleanly.
My question is: Why do I have to specify this encoding ? I thought that Perl adapted to its environment, and the localization environment variables should all be readable, right ?
Can someone explain the reason to me, or point me to relevant documentation ?
Thanks in anticipation.
EDIT: Ok, so, after reading through all these fine answers, I think I understand better my problems, and the solutions:
- I had probably tried a Perl version which DID take upon itself to automatically convert some streams, but not some others, and that had left me confused
- I'm pretty sure, by now, that Perl "decodes" input from Latin-1, and "encodes" output to Latin-1, by default.
- I've discovered a fine command-line switch, '-C' (you may read about it in the 'perlrun' documentation), which does exactly what I would have expected Perl to do, if called with the parameter 'L'. That is, '-CL' should decode STDIN from UTF-8, or encode STDOUT / STDERR to UTF-8, if such an encoding is mentioned in the 'LC_ALL', 'LC_TYPE', or 'LANG' environment variables.
- Of course, using explicit encoding (or explicitly stating in the script that we want it autodetected) is best practice, even if a bit more cumbersome to write
Please feel free to correct me if you feel that my conclusions are false or misleading.