in reply to Re: UTF-8 representation question
in thread UTF-8 representation question

Thank you, ikegami, for your extremely informative reply. I see your point about not notifying perl I was using UTf-8. When I include the 'use utf8' (without the semicolon) in front of my $test assignment line, the length() function still tells me that the length of $test is 2 chars, but it does print out correctly to the shell as the registered trademark symbol, instead of junk. If I insert the semicolon, it prints junk. Is it possible that my shell is interpreting the output differently than perl? I suppose I should read up on Perl's inner workings. One minor note however, the code I posted was in fact the code that I ran, so I am still confused on that point. I appreciate your help.

Replies are listed 'Best First'.
Re^3: UTF-8 representation question
by ikegami (Patriarch) on Sep 04, 2008 at 05:38 UTC

    Is it possible that my shell is interpreting the output differently than perl?

    It's all bytes until you pass them to something that cares, such as lc or your shell. Each of those "somethings" will interpret the bytes as it sees fit.

    One minor note however, the code I posted was in fact the code that I ran, so I am still confused on that point. I appreciate your help.

    There are a number of stetps between the file and Perlmonks where the bytes could have been substituted, plus PerlMonks itself and my browser.

    If I insert the semicolon, it prints junk.

    So far, we've only covered decoding the source. Sounds like you didn't properly encode the data while printing it. One way:

    #!/usr/bin/perl use strict; use warnings; # Decode source from UTF-8. use utf8; # Decode STDIN as per locale. # Encode STDOUT & STDERR as per locale. use open qw( :std :locale ); my $test = '...'; print($test); Dump($test);

    Or if you want to decode/encode your input/output using a specific encoding, you can do it as follows:

    #!/usr/bin/perl use strict; use warnings; # Decode source from UTF-8. use utf8; # Expect UTF-8 from STDIN. # Send UTF-8 to STDOUT & STDERR. BEGIN { binmode STDIN, ':encoding(UTF-8)' or die; binmode STDOUT, ':encoding(UTF-8)' or die; binmode STDERR, ':encoding(UTF-8)' or die; } my $test = '...'; print($test); Dump($test);

    Feel free to replace "..." with characters of your choice. If you're still having problem, please provide the Dump output, a description of what you see from the print (primarily the number of characters you see), how many characters you are expecting to see, and which of the two programs produced the output.