Re^3: javascript encodeURI() -> perl uri_unescape

The thing you are referring to as "two sequences" is actually the two-byte sequence for the utf-8 encoded character U00E5. (updated the grammar slightly to make more sense)

Naturally, we'd love to have an elegant and concise way to interpret this correctly as utf8 text, but I don't know enough about the URI modules to provide much guidance in that direction.

So instead, I'll offer an ad-hoc (but still somewhat concise) work-around -- it's a kluge, but it should work until you or some other monk can find the needed gems in the appropriate module(s):

use Encode;

# ... get the uri string into $_ by whatever means ...

$_ = "a%20%C3%A5%20%2F";

# first, let's turn the uri encoded string (with "%HH" for some bytes)
+ into binary:

s/\%([0-9a-f]{2})/chr(hex($1))/egi;

# then, since this produces a utf-8 byte sequence, let's "decode" that
+ into utf-8

$_ = decode( 'utf8', $_ );

# $_ now has utf8 flag set, and contains the string with expected unic
+ode characters

binmode STDOUT, ":utf8";

print;
[download]

The "binmode STDOUT" thing could be taken out if you add a "-CO" flag on the shebang line, I believe -- that "perlrun" option does the same thing as 'binmode STDOUT, ":utf8";'.

Comment on Re^3: javascript encodeURI() -> perl uri_unescape_utf8() ? Download Code

Replies are listed 'Best First'.
Re^4: javascript encodeURI() -> perl uri_unescape_utf8() ? by blahblahblah (Priest) on Dec 15, 2004 at 03:26 UTC
I think that CGI's 'unescape' method does the same thing. `use CGI qw(unescape); $_ = "a%20%C3%A5%20%2F"; $value = CGI::unescape($_);` [download] (it's in CGI/Util.pm)	[reply] [d/l]
Re^4: javascript encodeURI() -> perl uri_unescape_utf8() ? by nkropols (Sexton) on Dec 15, 2004 at 14:31 UTC
Thanks for your suggestion. This does not solve my problem, unfortunately. I have found that the decode function in the Encode module should be able to do the job. `use Encode; print decode('utf8', "\xC3\xA5");` [download] This prints the character I need. The problem now is: How to go from %C3%A5 etc. to \xC3\xA5? I tried `use Encode; $_='%C3%A5'; s/%/\\x/g; eval { $_=$_}; print decode('utf8', $_);` [download] but it does not work as expected. :-) This was actually solved in the previous post by graff.	[reply] [d/l] [select]
Re^5: javascript encodeURI() -> perl uri_unescape_utf8() ? by graff (Chancellor) on Dec 15, 2004 at 15:32 UTC
The problem now is: How to go from %C3%A5 etc. to \xC3\xA5? That's exactly what this line of code does: `s/\%([0-9A-Z]{2})/chr(hex($1))/egi;` [download] Did you try that? The left side matches any "%HH" pattern, and captures the two hex digits into $1; in the right side, "hex()" interprets $1 as a hex number, and "chr()" turns that value into a single-byte character. (Or it should, unless maybe you've put a "use utf8" somewhere above this line -- if that's the problem, put this regex into its own block with "use bytes": `{ use bytes; s/\%([0-9A-Z]{2})/chr(hex($1))/egi; }` [download] As for your eval attempt, I think the first "$_" in "$_=$_" needs to have a backslash in front of the dollar-sign: "\$_ = $_". In any case, I would advise against changing every percent-sign into "\x" -- there might be some cases of "%" that are not followed by a pair of hex digits. (update: Actually, in the context of dealing with uri strings, that last point is moot -- it surely must be the case in uri encoding that every "%" character is part of a "%HH" expression, and "%25" is used to express a literal percent-sign in the data. But the point is relevant for any data that might be a "defective" combination of uri encoding and plain text.)	[reply] [d/l] [select]