Maybe I'm better off forcefully converting all input and output to UTF-8

Yes. For many reasons, it is best to decode all inputs, and encode all output.

I still feel this is a bug in Perl, though.

I believe Perl doesn't support multi-byte locales (e.g. UTF-8).

Effort is placed on Unicode instead instead of adding to the locale system.

Is there a way – perhaps debugging argument – to see what \w applies to?

perlre: Match a "word" character (alphanumeric plus "_").

The following are equivalent:

( No, this is wrong ) 

/\w/                   # When no locale, when not restricted to ASCII
/\p{Word}/
/[_\p{Alnum}]/
/[_\p{Alphabetic}\p{Nd}]/

[download]

Derived property "Alphabetic". (100,520 codepoints in Perl 5.12.2)
Unicode character category "Nd". (411 codepoints in Perl 5.12.2)

Actual lists vary by version of Unicode and thus by version of Perl.

In reply to Re^3: use locale broken? (\w) by ikegami
in thread use locale broken? by december

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.