Re^2: uparse - Parse Unicode strings [sanity test]

Just as something of a sanity test for me, and perhaps a test you could try for yourself, here's uparse with its argument taken from three different sources not associated with PerlMonks.

$ perlbrew switch perl-5.32.0

$ perl -v | head -2 | tail -1
This is perl 5, version 32, subversion 0 (v5.32.0) built for cygwin-th
+read-multi
[download]

Copy-pasted from the Unicode PDF code chart "C1 Controls and Latin-1 Supplement (Range: 0080-00FF)":

$ uparse äöü

============================================================
String: 'äöü'
============================================================
ä       U+E4     LATIN SMALL LETTER A WITH DIAERESIS
ö       U+F6     LATIN SMALL LETTER O WITH DIAERESIS
ü       U+FC     LATIN SMALL LETTER U WITH DIAERESIS
------------------------------------------------------------

Generated directly from a perl command:

$ uparse `perl -C -e 'print "\x{e4}\x{f6}\x{fc}"'`

============================================================
String: 'äöü'
============================================================
ä       U+E4     LATIN SMALL LETTER A WITH DIAERESIS
ö       U+F6     LATIN SMALL LETTER O WITH DIAERESIS
ü       U+FC     LATIN SMALL LETTER U WITH DIAERESIS
------------------------------------------------------------

Generated separately then copy-pasted as an argument to uparse:

$ perl -C -e 'print "\N{LATIN SMALL LETTER A WITH DIAERESIS}\N{LATIN SMALL LETTER O WITH DIAERESIS}\N{LATIN SMALL LETTER U WITH DIAERESIS}"'
äöü

$ uparse äöü

============================================================
String: 'äöü'
============================================================
ä       U+E4     LATIN SMALL LETTER A WITH DIAERESIS
ö       U+F6     LATIN SMALL LETTER O WITH DIAERESIS
ü       U+FC     LATIN SMALL LETTER U WITH DIAERESIS
------------------------------------------------------------

— Ken

Comment on Re^2: uparse - Parse Unicode strings [sanity test] Select or Download Code

Replies are listed 'Best First'.
Re^3: uparse - Parse Unicode strings [sanity test] by jo37 (Curate) on Nov 20, 2023 at 11:14 UTC
Hi Ken! I found the reason for the strange behaviour: I didn't even remember, but I have PERL_UNICODE=SDAL set. Without this variable the script works correctly. More specifically, it's the "A" in it. From perlrun: `A 32 the @ARGV elements are expected to be strings encoded in UTF-8` [download] Thank you very much for your investigations! Greetings, -jo `$gryYup$d0ylprbpriprrYpkJl2xyl~rzg??P~5lp2hyl0p$`	[reply] [d/l]

Replies are listed 'Best First'.

Re^3: uparse - Parse Unicode strings [sanity test]
by jo37 (Curate) on Nov 20, 2023 at 11:14 UTC

Hi Ken!

I found the reason for the strange behaviour: I didn't even remember, but I have PERL_UNICODE=SDAL set. Without this variable the script works correctly. More specifically, it's the "A" in it. From perlrun:

A    32   the @ARGV elements are expected to be strings encoded
          in UTF-8
[download]

Thank you very much for your investigations!

Greetings,
-jo

$gryYup$d0ylprbpriprrYpkJl2xyl~rzg??P~5lp2hyl0p$

[reply]
[d/l]