in reply to Re: Weird file type problems transferring from Windows to Mac OS
in thread Weird file type problems transferring from Windows to Mac OS

Choroba

Thanks for the debugging tip.

Running hexdump -n NNN BAD.txt | xxd -r | file -, with NNN up to 1506 the result is text, at 1508 gives data.

Running hexdump -n 1520 BAD.txt I get:
0000500 55 00 c7 00 c3 00 4f 00 20 00 2e 00 2e 00 2e 00 0000510 2e 00 2e 00 2e 00 2e 00 2e 00 2e 00 2e 00 2e 00 * 00005e0 2e 00 20 00 34 00 20 00 20 00 0d 00 0a 00 32 00 00005f0

strange that the weird character appears before 1508, though ...

Replies are listed 'Best First'.
Re^3: Weird file type problems transferring from Windows to Mac OS
by Jim (Curate) on May 28, 2013 at 17:15 UTC

    Try using iconv instead of dos2unix to convert the Unicode character encoding scheme of the files. The gremlins in the text might be something like improperly unpaired UTF-16 surrogate characters. I would expect iconv to handle these anomalies better than dos2unix. (It should at least warn you about them in the default case.)

    Just a thought…

    UPDATE: Another good tool for diagnosing peculiar and elusive character encoding problems is BabelPad.

      BabelPad is for Windows only, unfortunately ...

        Did you try using iconv instead of dos2unix? If so, did it work?