There are two types of string variables in Perl. One type contains text, the other contains bytes.

Files contain bytes. If you want to write a text string to a file, the string needs to be converted from text to bytes. That's what the >:encoding(UTF-8) does.

OTOH if you want to read data from a file, without the :encoding(UTF-8) the string will contain bytes, and with it the string contains text.

Sad thing is, you can't reliably see from looking at a string if it's text or bytes. And if you mix the two up, you will see some broken output.

So if you use text strings internally in your program, you need the :encoding(UTF-8) both for reading and writing files, and you need to decode all other byte strings that come into your program (for example with %ENV or @ARGV).

OTOH some modules already decode strings for you (for example XML and JSON parsers), so you must be aware which module does that.

Perl 6 - second systems done right

In reply to Re^3: Mess with UTF-8, utf8 and raw encoding on live working platform by moritz
in thread Mess with UTF-8, utf8 and raw encoding on live working platform by AlfaProject

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.