I'm using it to debug a problem.

I recommend Devel::Peek's Dump.

The corruption is consistent with bytes with the high-bit set getting converted to their UTF8 encoding.

Way too many XS modules access a string's buffer with no regard to the setting of the UTF8 flag. Sounds like it's happening here.

string possible internal representations default typemap to char*
"\x{C9}" C9; UTF8=0 C9
C3,89; UTF8=1 C3,89

I suspect using utf::downgrade is the solution, but I want to verify that.

It is indeed the solution. utf8::downgrade converts the internal encoding of a string from utf8 to bytes if it isn't already. The XS module should do that for you, but you can do it for the module.

string possible internal representations downgraded default typemap to char*
"\x{C9}" C9; UTF8=0 C9; UTF8=0 C9
C3,89; UTF8=1 C9; UTF8=0 C9

In reply to Re^5: good way to implement utf8::is_utf8 for perl 5.8.0 by ikegami
in thread good way to implement utf8::is_utf8 for perl 5.8.0 by perl5ever

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.