"\x{feff}" does not map to cp1256, <IN> line 1. And the character
\x{feff} is displayed at the beginning of the file
FEFF is the unicode character code of the
BOM (Byte Order Mark).
You just have to ignore it (i.e skip over or remove it from the input).
(With UTF-8, the BOM has no real use (the byte order is always the
same), but on Windows the BOM is generally used to identify the file as
being unicode encoded.)
| [reply]
[d/l] |
| [reply] |
I guess your input file contains non-utf8 characters then. I think I was in a similar situation when I was mailed some utf8-encoded text documents and for some reason I don't know, there was a non-utf8 char at the very beginning. I guess the easiest way to go is to call getc(IN) just before the loop. This would assume though that there indeed is an invalid character there - adding some tests on the return value of getc may be necessary if you're not sure.
...I could also be totally wrong and your input file is OK and the problem is somewhere else.
update:
FEFF is the unicode character code of the BOM (Byte Order Mark).
I said wrongly that there is a non-UTF8 character (meaning non-utf8 byte sequence) and I was wrong. Of course, according to the error message, the unicode character only has no equivalent in cp-1256. Thanks almut for a proper explanation.
| [reply]
[d/l] |
Thanks ^_^ now it works fine. It is very strange, I can't find anything wrong with the text, but your suggestion seems to be correct however.
| [reply] |
