http://qs1969.pair.com?node_id=11133090

derion has asked for the wisdom of the Perl Monks concerning the following question:

I have some XML code that I get with LWP::UserAgent.
The XML is declared and seems to contain ISO-8859-1 data with the header:
<?xml version="1.0" encoding="ISO-8859-1"?>...
I used the following code to get the data:
my $url = 'https://some.url/file.xml'; my $LWP_Data; use LWP::UserAgent; $LWP_Data->{ua} = LWP::UserAgent->new; $LWP_Data->{ua}->timeout(7); $LWP_Data->{feed} = $LWP_Data->{ua}->get($url); if ($LWP_Data->{feed}->is_success) { my $xml = $LWP_Data->{feed}->content; use XML::Simple qw(:strict); my $ref = XMLin( $xml, ForceArray => 1,KeyAttr => [ ]); ...

The results of $ref shows corrupted characters, e.g. ΓΌ instead of ü which made me think that the original data might not be ISO-8859-1 as declared. After some testing I assume that XML::Simple simply does not care about the declared encoding because the following works.

my $url = 'https://some.url/file.xml'; my $LWP_Data; use LWP::UserAgent; $LWP_Data->{ua} = LWP::UserAgent->new; $LWP_Data->{ua}->timeout(7); $LWP_Data->{feed} = $LWP_Data->{ua}->get($url); if ($LWP_Data->{feed}->is_success) { my $xml = $LWP_Data->{feed}->content; use Encode; my $encoded_xml = Encode::encode_utf8($xml); use XML::Simple qw(:strict); my $ref = XMLin( $encoded_xml, ForceArray => 1,KeyAttr => [ ]); # for a not nested hash ref foreach my $key (keys %{$ref }) { Encode::from_to($ref ->{$key}, "UTF-8", "iso-8859-1"); } ...

What I found in the forum is: Character Conversion Conundrum (https://perlmonks.org/?node_id=416914) which kind of implies XML::Simple converts to UTF-8 while it is not working when I only convert the results from utf-8 to iso-8859-1.

I am curious if what I assume is correct, if I might have read somewhere what I trialed and errored or if I am on the wrong track with my assumption.

Thanks for your feedback.
Regards
derion