in reply to Diiference between these two filenames / strings

So there's a few complicating factors here. First you need to understand how character encoding works. For that I would suggest a thorough reading of https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character-sets-no-excuses/.

With that in mind, your source code files are usually saved in UTF-8, and strings that come into your program through filehandles or command line arguments also come in as UTF-8, unless you set layers that decode them. The source code itself is automatically decoded if you "use utf8;". With input strings you can use Encode to decode them if not the ":encoding(UTF-8)" layer on the handle. It's usually a good idea to work with decoded strings because then logically your string contains the characters you think it does, instead of the bytes that represent them in UTF-8.

But the problem is that you're dealing with filenames here, and filenames are all sorts of broken in Perl.* Perl essentially treats them as their *internal* bytes like a buggy XS module might, regardless of what the *logical* contents of the string are. So that's why there is a difference here when you didn't do anything wrong. The first string can't be represented in your native encoding, so the internal bytes are accidentally the correct UTF-8 encoding of your filename. The second string can, so the internal bytes are not the same as the UTF-8 encoding, unless you utf8::upgrade it to change the internal encoding, or use Encode to explicitly change its logical contents to the UTF-8 encoding of the string. My recommendation would be: use decoded strings in general, and encode them explicitly for use as a filename.

* https://rt.perl.org/Public/Bug/Display.html?id=130831

  • Comment on Re: Diiference between these two filenames / strings