Re^2: File::Find::Rule returns different filenames if they have chars with accents: OSX vs Linux

I have searched on normalisation and OSX FS and it seems it is a fine mess. See e.g. https://eclecticlight.co/2021/05/08/explainer-unicode-normalization-and-apfs/ . Oh well the unicode curse got apple who thought they could tame it.

Apropos: I learned a new [sic] neologism ... dystextia.

Comment on Re^2: File::Find::Rule returns different filenames if they have chars with accents: OSX vs Linux Download Code

Replies are listed 'Best First'.
Re^3: File::Find::Rule returns different filenames if they have chars with accents: OSX vs Linux by kikuchiyo (Hermit) on Jan 03, 2024 at 14:48 UTC
A fine mess indeed. I've experimented as well, and on linux it's possible to have two (or more) different files whose names appear to be the same because of different unicode normalization. `perl -e 'for (["a\x{0301}", "decomposed"], ["\xc3\xa1", "composed"]) {open my $F, ">", $_->[0]; print $F $_->[1]; close $F}'` After running this: `$ ls -rw-r--r--. 1 user user 8 Jan 3 15:31 á -rw-r--r--. 1 user user 10 Jan 3 15:31 á` [download] (in the terminal these appear identical) If I turn on my native keyboard layout and type "less á", I get the composed file. But I can copy and paste the decomposed string into the terminal, and access the other file as well. Apparently, this filesystem, and linux filesystems in general, don't assume and enforce much about file name encoding: file names are just a sequence of bytes, and the user can keep the pieces. My gut feeling is that Apple's way, that is, normalization, is "better" from an usability standpoint - but then it had better be completely consistent and enforced everywhere.	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re^3: File::Find::Rule returns different filenames if they have chars with accents: OSX vs Linux
by kikuchiyo (Hermit) on Jan 03, 2024 at 14:48 UTC

A fine mess indeed.

I've experimented as well, and on linux it's possible to have two (or more) different files whose names appear to be the same because of different unicode normalization.

perl -e 'for (["a\x{0301}", "decomposed"], ["\xc3\xa1", "composed"]) {open my $F, ">", $_->[0]; print $F $_->[1]; close $F}'

After running this:

$ ls
-rw-r--r--. 1 user user     8 Jan  3 15:31 á
-rw-r--r--. 1 user user    10 Jan  3 15:31 a&#769;
[download]

If I turn on my native keyboard layout and type "less á", I get the composed file. But I can copy and paste the decomposed string into the terminal, and access the other file as well.

Apparently, this filesystem, and linux filesystems in general, don't assume and enforce much about file name encoding: file names are just a sequence of bytes, and the user can keep the pieces.

My gut feeling is that Apple's way, that is, normalization, is "better" from an usability standpoint - but then it had better be completely consistent and enforced everywhere.

[reply]
[d/l]
[select]