Special character in file name -- die error messed up

leszekdubiel has asked for the wisdom of the Perl Monks concerning the following question:

Is there any way to make "die" handle polish file names better? It's all about chars in file name. See:

# cat ąćęłńó.pl 
#!/usr/bin/perl -CSDA

use utf8; 

die "language specific chars:  ąćęłńó"; 


# ./ąćęłńó.pl 
language specific chars:  ąćęłńó at ./Ä
ÄÄÅÅ
    Ã³.pl line 5.

Comment on Special character in file name -- die error messed up

Replies are listed 'Best First'.
Re: Special character in file name -- die error messed up by haj (Vicar) on May 28, 2020 at 11:54 UTC
This is a bit tricky and a known shortcoming of Perl which isn't all that easy to fix. To start with, Perl does not know which encoding the file system applies to represent polish characters, and you have no way of specifying it. So, the name of your source file is known to Perl as bytes, encoded in whatever encoding the file system chooses. Nowadays, unless you are on Windows, UTF-8 is a reasonable guess, but Perl does not guess. Your command line parameter `-CS` then enforces UTF-8 encoding of STDERR, which is fine for the characters in your error message but not for file names. So, you get the file name doubly encoded. One way to get out of this is to apply encoding yourself, and drop the command line parameters. This sort of works, but makes the assumption that the file name is encoded in UTF-8: `#!/usr/bin/perl # saved as ąćęłńó.pl use utf8; use Encode qw(encode); die encode('UTF-8',"language specific chars: ąćęłńó");` Edited because I forgot (again) that the code tags break non-ASCII characters	[reply]
Re: Special character in file name -- die error messed up by soonix (Chancellor) on May 28, 2020 at 12:30 UTC
This is independent of `use utf8;` On my Linux system, I don't have this problem, on Windows I could solve it by switching to Code page 65000 (!) see https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/chcp and/or CPAN module Win32::Codepage (!) - which is the code page for UTF-7… Update: I hadn't taken the -C flag into account. With -CS, I also get wrong encoding on Linux. So haj's answer is it. Update 2:with chcp 65001, it also works with -CS on Windows	[reply] [d/l]
Re: Special character in file name -- die error messed up by leszekdubiel (Scribe) on May 28, 2020 at 20:41 UTC
Tanks for help. I will live with that inconvenience.	[reply]