Re: How to handle unicode txt file on Windows

Replies are listed 'Best First'.
Re^2: How to handle unicode txt file on Windows by ikegami (Patriarch) on Nov 03, 2008 at 20:38 UTC
And `:encoding(UTF-16)` will work even better since it absorbs the BOM.	[reply] [d/l]
Re^3: How to handle unicode txt file on Windows by cheerful (Initiate) on Nov 03, 2008 at 21:36 UTC
What would happen if it's called on a non-unicode file?	[reply]
Re^4: How to handle unicode txt file on Windows by ikegami (Patriarch) on Nov 03, 2008 at 21:51 UTC
Using `:encoding(UTF-16)` on a file not encoded using UTF-16 will likely die as a result of not finding \xFE\xFF or \xFF\xFE at the start of the file. Using `:encoding(UTF-16le)` on a file not encoded using UTF-16le will result in garbage and/or die. You must know the encoding of the file to be able to read it. I think File::BOM will help you. It imposes an limitation (the presence of a BOM for non-default encodings), but that doesn't sound like a problem for you.	[reply] [d/l] [select]