Problem with url escaping in Pod::Simple::HTML and UTF-8

xenu has asked for the wisdom of the Perl Monks concerning the following question:

Hi! In the following code there's problem with URL escaping:

#!/usr/bin/perl
use strict;
use warnings;
use utf8;
use Pod::Simple::HTML
my $pod_html_output;
my $content = <<CONTENT;
=head1 foobar

L<<utf-8 letter (a with ogonek) which i can't paste here because perlm
+onks doesn't support unicode>>>

=cut
CONTENT
my $pod_html = Pod::Simple::HTML->new;
$pod_html->output_string(\$pod_html_output);
$pod_html->parse_string_document($content);
print $pod_html_output;
[download]

'<utf-8 letter (a with ogonek) which i can't paste here because perlmonks doesn't support unicode>' is escaped to '%25105' instead of '%C4%85' I have tried overriding general_url_escape() but then i got even stranger results:

{
    package Foobar::Pod::Simple::HTML;
    use base qw(Pod::Simple::HTML);
    use CGI::Util qw(escape);
    sub general_url_escape {
        my($self, $string) = @_;
        $string = escape($string);
        return $string;
    }
}
[download]

In this case '<utf-8 letter (a with ogonek) which i can't paste here because perlmonks doesn't support unicode>' was escaped to '%105' (wtf?). Thanks in advance for help.

Comment on Problem with url escaping in Pod::Simple::HTML and UTF-8 Select or Download Code

Replies are listed 'Best First'.
Re: Problem with url escaping in Pod::Simple::HTML and UTF-8 by choroba (Cardinal) on Jun 26, 2011 at 11:11 UTC
A semicolon is missing after `use Pod::Simple::HTML`. I also get this warning when running your (fixed) code: `Malformed UTF-8 character (unexpected continuation byte 0xbe, with no +preceding start byte) at 911435.perl line 10.` [download] Update: This warning remains even if I only keep this: `#!/usr/bin/perl use strict; use warnings; use utf8; my $content = <<CONTENT; =head1 foobar L<<utf-8 letter (a with ogonek) which i can't paste here because perlm +onks doesn't support unicode>>> =cut CONTENT` [download]	[reply] [d/l] [select]
Re: Problem with url escaping in Pod::Simple::HTML and UTF-8 by Anonymous Monk on Jun 26, 2011 at 11:41 UTC
utf-8 letter (a with ogonek) which i can't paste here because perlmonks doesn't support unicode :) Perl is perl :) So U+0104 Ą Latin Capital Letter A with ogonek `print <<"__UTF__"; \N{U+0104} \x{0104} __UTF__ ## this needs ## use charnames ':full'; ## \N{Latin Capital Letter A with ogonek}` [download]	[reply] [d/l]
Re: Problem with url escaping in Pod::Simple::HTML and UTF-8 by Anonymous Monk on Jun 26, 2011 at 12:34 UTC
(wtf?). Thanks in advance for help. I can confirm this, there is like ~3 bugs here. Adding `=encoding utf8` [download] makes ::HTML die with `Cannot decode string with wide characters at C:/perl/5.12.2/lib/MSWin3 +2-x86-multi-thread/Encode.pm line 176.` [download] Pod::Simple::XHTML generates `<p><a href="http://search.cpan.org/perldoc?&#x104;">Ą</a></p +>` [download] Both ::HTML and ::XHTML generate `<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1 +" />` [download] regardless of BOM or `=encoding utf8` [download] which doesn't kill ::XHTML This is probably related Bug #24820 for Pod-Simple: Pod encoding, unicode issues You should report bug upstream :)	[reply] [d/l] [select]
Re: Problem with url escaping in Pod::Simple::HTML and UTF-8 by xenu (Novice) on Jun 26, 2011 at 14:56 UTC
Problem solved by Rhomboid@reddit here.	[reply]
Re^2: Problem with url escaping in Pod::Simple::HTML and UTF-8 by Corion (Patriarch) on Jun 26, 2011 at 15:02 UTC
If you are crossposting, please at least have the courtesy to say so in your original post. I consider it rude to crosspost questions without telling people where else you have asked the question.	[reply]