Re^3: How to handle unicode txt file on Windows

Replies are listed 'Best First'.
Re^4: How to handle unicode txt file on Windows by ikegami (Patriarch) on Nov 03, 2008 at 21:51 UTC
Using `:encoding(UTF-16)` on a file not encoded using UTF-16 will likely die as a result of not finding \xFE\xFF or \xFF\xFE at the start of the file. Using `:encoding(UTF-16le)` on a file not encoded using UTF-16le will result in garbage and/or die. You must know the encoding of the file to be able to read it. I think File::BOM will help you. It imposes an limitation (the presence of a BOM for non-default encodings), but that doesn't sound like a problem for you.	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re^4: How to handle unicode txt file on Windows
by ikegami (Patriarch) on Nov 03, 2008 at 21:51 UTC

Using :encoding(UTF-16) on a file not encoded using UTF-16 will likely die as a result of not finding \xFE\xFF or \xFF\xFE at the start of the file.

Using :encoding(UTF-16le) on a file not encoded using UTF-16le will result in garbage and/or die.

You must know the encoding of the file to be able to read it.

I think File::BOM will help you. It imposes an limitation (the presence of a BOM for non-default encodings), but that doesn't sound like a problem for you.

[reply]
[d/l]
[select]