Confused about using Entities.pm

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Ok, I'm a bit confused here.

I am trying to create a feed for an iPhone app but I need utf8 values for the title field of the feed and html entities for the description field of the feed.

I am reading a few feeds and compressing them into on larger feed that the app reads. The description is already encoded into it's html entities, so that part is done.

However, the title needs to be converted to utf-8 as there are many latin characters (accute, etc) for our Spanish content. I use the decode_entities function from the Entities.pm lib, but I'm noticing that the char values that it converts to are breaking my feed -- what I mean is that the feed is not valid, even though there are CDATA tags around the title field. Upon further investigation, I've discovered that this lib is converting from an html entity to it's corresponding unicode value but I think it is using the ISO-8859 instead of utf8.

What am I doing wrong? How do I convert from an html-entity to utf8? I actually don't need the feeds to be valid -- my objective-c parser has no problem reading the feed as ascii, etc. but I need others to use this feed and need them to be valid.

Comment on Confused about using Entities.pm

Replies are listed 'Best First'.
Re: Confused about using Entities.pm by ikegami (Patriarch) on Dec 14, 2010 at 00:12 UTC
I'm guessing you didn't actually encode the text you place in your feed. `$ perl -MHTML::Entities -MXML::LibXML -E' $xml="<root>".decode_entities("é")."</root>"; utf8::encode($xml); XML::LibXML->new->parse_string($xml); say "ok"; ' ok $ perl -MHTML::Entities -MXML::LibXML -E' $xml="<root>".decode_entities("é")."</root>"; #utf8::encode($xml); XML::LibXML->new->parse_string($xml); say "ok"; ' :1: parser error : Input is not proper UTF-8, indicate encoding ! Bytes: 0xE9 0x3C 0x2F 0x72 <root>?</root> ^` [download]	[reply] [d/l]

Replies are listed 'Best First'.

Re: Confused about using Entities.pm
by ikegami (Patriarch) on Dec 14, 2010 at 00:12 UTC

$ perl -MHTML::Entities -MXML::LibXML -E'
   $xml="<root>".decode_entities("&eacute;")."</root>";
   utf8::encode($xml);
   XML::LibXML->new->parse_string($xml);
   say "ok";
'
ok

$ perl -MHTML::Entities -MXML::LibXML -E'
   $xml="<root>".decode_entities("&eacute;")."</root>";
   #utf8::encode($xml);
   XML::LibXML->new->parse_string($xml);
   say "ok";
'
:1: parser error : Input is not proper UTF-8, indicate encoding !
Bytes: 0xE9 0x3C 0x2F 0x72
<root>?</root>
      ^
[download]

[reply]
[d/l]