If I understand the situation correctly (not sure that I do), utf8 data is coming in on a CGI parameter string (or something like that), but at the point where the utf8 string is assigned to a scalar in the script, that scalar is not being flagged as holding utf8 data. In this case, the safer, more stable way (relative to what liz suggested above) to make perl use it as utf8 is to "decode" it: 
  my $utf8_param = decode( 'utf8', $input_param );

[download]
and maybe add the third arg to the decode call, so that it does something useful in case the input data turns out not to be valid utf8. See the section of the Encode man page headed "Handling Malformed Data": you can pass a "CHECK" parameter that says "die on error", and wrap the decode call in an eval block to see whether it worked, before moving on to doing regex matches involving unicode character classes.

(If you don't check the result of the decode call, parts that it couldn't decode will show up as "\x{FFFD}", which would be "safe", but not very informative in terms of diagnostics.)

In reply to Re: Unicode and Forms by graff
in thread Unicode and Forms by mdxi

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.