I would agree with GrandFather's opinion: most likely your hash keys are being saved as UTF-16BE (big-endian), though I have no idea why that is happening. I would expect that this affects the hash values as well (and the line-termination, whether it's LF or CRLF).

You don't see the null bytes when using "more", because your terminal display ignores them.

If you show some code that demonstrates how this file was written, we could show you ways to avoid the problem. In the meantime, you might try passing that data file through a one-liner, like this: 

perl -pe 'BEGIN{binmode STDIN,"encoding(UTF-16BE)"; binmode STDOUT,"ut
+f8"}' < file.in > file.out

[download]
But first you should follow the other advice given above: inspect the file more carefully with a hexdump tool, to confirm whether or not UTF-16 is being used regularly for every character in the file (not just the hash keys). If my guess is wrong, and only the hash keys are 16-bit characters, the script above would screw things up badly, and a slightly more complicated script would be needed to fix the file.

In reply to Re: unreadable hash keys by graff
in thread unreadable hash keys by nujgnahz

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.