in reply to Re^3: UTF8 versus \w in pattern matching
in thread UTF8 versus \w in pattern matching

Thanks. Both pass here as well. 

1..2
ok 1 - Matched
ok 2 - Capture group 1

[download]

I've double-checked the source files, which are from WordPress via HTTP, and they get identified by the 'file' utility as "HTML document, UTF-8 Unicode text".

Replies are listed 'Best First'.
Re^5: UTF8 versus \w in pattern matching
by hippo (Archbishop) on Jul 06, 2021 at 12:20 UTC

    So there's nothing wrong with your version of perl and it correctly matches the UTF-8 accented characters with \p{Word}, and presumably also with \w if you change the value of $re thus: my $re = qr/^([\/\w]+)/;

    Are you definitely decoding the contents of these files when you read them in your perl script?

    Might also be worth checking the actual data in the data files with eg. hexdump.

    🦛

[reply]
[d/l]
[select]

      Using the formula my $re = qr/^([\/\w]+)/; as the pattern has the same problems. I am quite sure that the input files are UTF-8. However, checking in different terminals, the script renders properly if I change the terminal to ISO-8859-15 away from UTF-8, even with \N{LATIN SMALL LETTER A WITH ACUTE} for the letters. So this may be a terminal problem, except I really wonder why the script, which has \N{LATIN SMALL LETTER A WITH ACUTE} is still outputting ISO-8859-15 instead of UTF-8.

[reply]
[d/l]
[select]
        Using the formula my $re = qr/^([\/\w]+)/; as the pattern has the same problems.

        For clarity, the test script which I provided works just as well with this regex. The point is that it demonstrates that there is nothing wrong with your perl code which does the regex matching and therefore the only logical conclusion is that your data is not what you think it is.

        Are you decoding your UTF-8 data when you read it from the data files in your script? If not, that is the problem.

        If you can provide a real SSCCE then I'm sure all will become clear.

        🦛

[reply]
[d/l]

In Section Seekers of Perl Wisdom