in reply to Re: UTF8 versus \w in pattern matching
in thread UTF8 versus \w in pattern matching

Thanks. That part about the U+7F..U+FF range explains things. I see that m/^([\/\-\_\.\p{Word}\x7f-\xff]+)$/ matches, and m/^([\/\-\_\.\p{Word}]+)$/ does not. I presume that is because the data upstream might really be ISO-8859-15 and not UTF-8? Should I try to convert the U+7F..U+FF range into UTF-8 before further processing? If so, how?

Replies are listed 'Best First'.
Re^3: UTF8 versus \w in pattern matching
by ikegami (Patriarch) on Jul 11, 2021 at 05:12 UTC

    Read again. The part about U+7F..U+FF applies to \w, but not to \p{Word}.

    # Sometimes \w matches U+E9. $ perl -Mfeature=say -e'say "\xE9" =~ /^\w/ || 0' 0 # Sometimes it doesn't. $ perl -Mfeature=say -e'say "\xE9\x{2660}" =~ /^\w/ || 0' 1 # \w always matches U+E9 with "use 5.014;". $ perl -Mfeature=say -e'use 5.014; say "\xE9" =~ /^\w/ || 0' 1 # \p{Word} always matches U+E9, period. $ perl -Mfeature=say -e'say "\xE9" =~ /^\p{Word}/ || 0' 1

    Your questions make absolutely no sense since you're not using \w. And I said the fix for that bug was to add use 5.014;, not to convert the input.

    Seeking work! You can reach me at ikegami@adaelis.com