in reply to Re: Lost in encoding in Twig
in thread Lost in encoding in Twig

The recommendations by pcouderc are the usual ones, and I would actually code the script that way. But the fact is that when I run your script as originally posted and cat the output I get '<myxml fille="clémence"/>'. Have you looked at xml.out with 'hexdump -C'? If you see 'clémence' encoded as '63 6c c3 a9 6d 65 6e 63 65' it means your Perl output is correct. But if your command shell is set to interpret output as ISO Latin-1 you would get 'clémence' even if your script is doing the right thing.

Replies are listed 'Best First'.
Re^3: Lost in encoding in Twig
by pcouderc (Monk) on Jan 09, 2015 at 13:37 UTC
    Thank you both.
    I do not think I need utf8 as my perl is v5.18 (debian jessie).

    Hexdump on my code.pl is correct c3 a9.
    Hexdump on my out.xml is incorrect and correspond to my cat : c3 83 c2 a9 (é).
    Sorry, I do not know how to check if my "command shell is set to interpret output as ISO Latin". But my locale is : LANG=en_US.UTF-8.
    I can use "keep-encoded" to make it work (in this case), but I would like to understand...
[reply]
      I do not think I need utf8 as my perl is v5.18 (debian jessie).

      That's fine (assuming you are correct). If you are relying on new features such as this in your code, it is probably a good idea to specify that minimum version of perl at the top so that this will trap any otherwise confusing errors should you (or someone else) try to run it on an older perl. eg:

      use 5.018;
      I can use "keep-encoded" to make it work (in this case), but I would like to understand...

      In a nutshell, to get unicode (and more specifically utf8) to work properly in perl you have to decode your inputs and encode your outputs. In the script here, you have no inputs to worry about because the data is hardcoded in your source. The output however does matter because you are printing this data to a file and so have to encode it first. You can do that manually, but it's generally considered a better idea to open the file with the :utf8 layer via binmode (or directly with open) so that it all gets encoded as it passes out through the I/O system. See also PerlIO::encoding.

[reply]
[d/l]
[select]
        Thank you,
        1- are you trying to tell me that utf8 is a "new" feature ? in 2014 ? sorry, in 2015 ;) .

        2- do you want to tell that for each operation in perl, and particularly for IO, I MUST specify utf8?
        Is there today another coding than utf8?

        3- Is not it possible to specify that all is utf8 (unless exceptions) at computer level ? or at least at perl level, or at least at script level ?

[reply]

In Section Seekers of Perl Wisdom