I have to say I had exactly the same problem trying to send UTF8 encoded XML stuff over HTTP (using Frontier::RPC2). Perl and me both got terribly confused over whether any given string was UTF8 or not.

This ended up being a complete nightmare especially since this code had to work on 5.5 and 5.6. In the end I worked out that something in Frontier::RPC2 was converting my strings to UTF8 behind my back under 5.6 whereas I had to convert them by hand in 5.5.

I experimented with 'use bytes' too but that didn't cut it either.

This code snipped might be useful 

$unicode_perl = 1;
{ local $SIG{__DIE__}; eval "use utf8; 1" or $unicode_perl = 0; }

[download]
Altogether a very unpleasant experience which makes me think that 5.6 isn't really ready for UTF :-( Maybe 5.8 will be better in this regard.

In reply to Re: What is the proper way to deal with Char v. Byte Semantics by ncw
in thread What is the proper way to deal with Char v. Byte Semantics by mongre345

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.