Ah, so it is the other way round, and very odd indeed. Runinng your code I get the same result, and I don't have UTF-8 in my LANG setting, either. It seems that HTML::TokeParser turns on the UTF-8 flag on strings returned by the get_text() method:
#!/usr/bin/perl use HTML::TokeParser; #use Data::Dump::Streamer; use strict; use Devel::Peek; local $/; my $lines = <DATA>; my $tok_par = HTML::TokeParser->new(\$lines); my $tok_inf = $tok_par->get_token ; my $tok_typ = shift @{$tok_inf}; my $title = $tok_par->get_text() || "<NO TITLE FOUND>"; Dump ($title); __DATA__ <title>egrave: &egrave; : eacute: &eacute; : rsquo: &rsquo; : lsquo: & +lsquo;</title> __END__ SV = PV(0x81b4290) at 0x81ed950 REFCNT = 1 FLAGS = (PADBUSY,PADMY,POK,pPOK,UTF8) PV = 0x8207b90 "egrave: \303\250 : eacute: \303\251 : rsquo: \342\20 +0\231 : lsquo: \342\200\230"\0 [UTF8 "egrave: \x{e8} : eacute: \x{e9} + : rsquo: \x{2019} : lsquo: \x{2018}"] CUR = 49 LEN = 52

- that's why you see the right output on your UTF-8 terminal at home, but garbled stuff on the servers terminal.

Hmm. I call that a bug :-)

--shmem

_($_=" "x(1<<5)."?\n".q·/)Oo.  G°\        /
                              /\_¯/(q    /
----------------------------  \__(m.====·.(_("always off the crowd"))."·
");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}

In reply to Re^5: HTML::TokeParser, get_text scrambling rsquo and lsquo by shmem
in thread HTML::TokeParser, get_text scrambling rsquo and lsquo by tridral

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.