If I change s/\r\n//g; to s/\r//g; then it prints everything ...

The substitution  s/\r\n//g changes all exact sequences of  \r\n to the empty string, and apparently there is no such sequence present for you say no change occurs. This might have worked better had you used the  [\r\n] character class instead. The substitution  s/\r//g removes all  \r (carriage-return) characters, which are, I think, the root of your problem.

The substituiton  s/[^[:ascii:]]//g removes all non-ASCII characters, but  \r and  \n are ASCII characters! The transliteration  tr/\x80-\xFF//d removes all 8-bit characters outside the ASCII range, but  \r \n are still ASCII characters.

Update: Adding code example:

c:\@Work\Perl\monks>perl -wMstrict -le "my $s = qq{ab-cd\tef\n\x80\x81\xaa\xab\xacfoo\rbar}; print qq{[[$s]] \n}; ;; $s =~ s/\r//g; print qq{[[$s]] \n}; ;; $s =~ s/[^[:ascii:]]//g; print qq{[[$s]] \n}; " [[ab-cd ef bar]] oo [[ab-cd ef Çü¬½¼foobar]] [[ab-cd ef foobar]]


In reply to Re: Dealing with non-ascii characters when reading file. by AnomalousMonk
in thread Dealing with non-ascii characters when reading file. by rjbioinf

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.