Converting string literal represetation of utf to utf8

djihed has asked for the wisdom of the Perl Monks concerning the following question:

I have some ascii strings output by a different program and written in a file. They contain utf-8 byte representation that I need to convert to utf8. For example, how to convert:

my $a = "\\xC4\\x80"; # 8 ascii characters
[download]

"\xC4\x80" # 2 code points representing a utf character.
[download]

A simple search and replace won't do, by the way.

Comment on Converting string literal represetation of utf to utf8 Select or Download Code

Replies are listed 'Best First'.
Re: Converting string literal represetation of utf to utf8 by dave_the_m (Monsignor) on Jun 17, 2011 at 10:56 UTC
Is this what you want: `my $a = "\\xC4\\x80"; $a =~ s/\\x([0-9A-Fa-f]+)/chr(hex($1))/ge; utf8::decode($a); printf "length=%d, ord=0x%x\n", length($a), ord($a);` [download] Outputs: `length=1, ord=0x100` [download] Dave.	[reply] [d/l] [select]
Re^2: Converting string literal represetation of utf to utf8 by djihed (Novice) on Jun 17, 2011 at 14:42 UTC
Yes, thanks.	[reply]
Re: Converting string literal represetation of utf to utf8 by moritz (Cardinal) on Jun 17, 2011 at 10:18 UTC
A simple search and replace won't do, by the way. Why not? What have you tried? Perl 6 - second systems done right	[reply]
Re^2: Converting string literal represetation of utf to utf8 by djihed (Novice) on Jun 17, 2011 at 10:27 UTC
What didn't work: `my $a = "\\xC4\\x80"; utf8::decode($a); # does not work, \\x are literal ascii. # is still the same. $a =~ s/\\x(\w\w)/\x$1/g; # Does not work. \x is interpreted withou +t the code points.` [download]	[reply] [d/l]
Re^3: Converting string literal represetation of utf to utf8 by moritz (Cardinal) on Jun 17, 2011 at 11:52 UTC
Well, at some point you need to make the actual conversion. For example `s/\\x(\w\w)/chr hex $1/ge` [download] Perl 6 - second systems done right	[reply] [d/l]