LWP::Simple is it get decoded or not ?

remiah has asked for the wisdom of the Perl Monks concerning the following question:

Hello monks.

Today I uploaded my perl program to my server. My local perl is 5.12.2 and server'perl is 5.8.8. And I met strange problem. LWP::Simple'get() function gives me decoded utf8 characters with my 5.12.2 local machine. But 5.8.8 machine gives me not decoded utf8 bytes. Each version of LWP::Simple is 5.835(local) and 5.810(server).

In get() function, LWP::Simple calls HTTP::Response's "decoded_content" method. Document of CPAN says

"This will return the content after any Content-Encoding and charsets have been decoded."

I guess there was some change for HTTP::Response's decoded _content. Does anyone met the same situation? Maybe,

$html=utf8::is_utf8($html) ? $html : decode('utf8', $html);
[download]

will solve my problem, but I would like to hear from monk's opinion.

regards.( I am sorry for my poor english)

Comment on LWP::Simple is it get decoded or not ? Download Code

Replies are listed 'Best First'.
Re: LWP::Simple is it get decoded or not ? by rpnoble419 (Pilgrim) on Aug 10, 2012 at 13:28 UTC
Uncode support changed in Perl 5.10 from the way it was handled in 5.8.8. Look at http://perldoc.perl.org/perlunicode.html for more information	[reply]
Re^2: LWP::Simple is it get decoded or not ? by remiah (Hermit) on Aug 10, 2012 at 22:52 UTC
thanks for reply rpnoble419. I will read through perluniintro, perlunicode anyway. This time, really. i became a little bit nervous for decoding, encoding way of each modules now. dbd::sqlite gives me decoded character if i set sqlite_unicode to 1, but they seems automatically encode characters to bytes when i put characters to database regardless of sqlite_unicode switch. it gives me no warnings when i put decoded characters to databases when i don't use sqlite_unicode. i thought dbd::ado gave me warning... wasn't it ? when i pass decoded characters to template toolkit, i pass binmode switch to proccess function. For "decoding" template file it self, I am using Template::Provider::Encoding. And now LWP::Simple... It seems I am stumbling all unicode stones.	[reply]
Re^2: LWP::Simple is it get decoded or not ? by remiah (Hermit) on Aug 11, 2012 at 04:07 UTC
It was version difference of LWP::Simple to get decoded character or not decoded bytes. 5.810 of LWP::Simple doesn't use decoded_content() but it uses content() method. On the other hand 5.835 uses decoded_content() method, so it returns decoded character. I decided to use useragent and decoded_content explicitly, to work with each enviornment. Below is test script with LWP::Simple 5.810 Read more... (3 kB) thanks a lot.	[reply] [d/l]