You guys are AWESOME! All your comments helped give me insight into this issue I've been pondering for a few months. Thanks to you guys I've found a solution! In particular, Joost mentioned that use bytes messes up utf8 encoding flags, and graff pointed me to the perl docs more utf8 byte thinking. Somehow this time this got me thinking in a different direction which led me to this:
http://ahinea.com/en/tech/perl-unicode-struggle.html.
In particular, check out this:
$ustring2 = pack "U0C*", unpack "C*", $ustring2;

Whenever I did $string = chr(20000), the length function counted 1 character. When I had input from a form, the length function counted in bytes. Somehow my form input function was messing up the flags.

Apparently use bytes may have been working within scope, but the string we were analyzing had messed up utf8 flags. Putting a no bytes before and a use bytes after my few lines of counting characters keeps everything working super.

So why do we want to use bytes? Our application counts the number if bytes in a message, wraps it in a wrapper, and tells some other component how many bytes to expect. If the count is wrong the message is rejected and components get shut down. We were uploading files with Chinese characters in the filename and the application seemed to shut down. We traced this problem to the length function counting characters instead of bytes. Since most string functions we use deal with byte counts, we'd like to use bytes and make character counts the exception.

In reply to Re^2: Another use bytes and length issue by zubster
in thread Another use bytes and length issue by zubster

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.