John M. Dlugosz has asked for the wisdom of the Perl Monks concerning the following question:

Working in Windows, I was used to Perl normalizing the end-of-line sequence to a \n even if it was \r\n on the disk.

Under Linix, I'm seeing \r's left on the line after I chomp them. Since files may come from HTTP or other network stuff that uses \r\n naturally, I should handle it.

How do I do a chomp saying "remove any style of line ending from this line, whatever it may be"?

Thanks
John

Replies are listed 'Best First'.
Re: \r\n at end of line (\s)
by tye (Sage) on Apr 19, 2011 at 02:36 UTC

    It is always a mistake to make trailing whitespace significant. So don't use chomp, use s/\s+$// instead. This has the side benefit of also preventing you from suffering from "\r"s.

    - tye        

[reply]
[d/l]
      Unless you are looking for a line that contains }}} with no trailing whitespace. I expected $ to match the eol.
[reply]

      tye

      If you use this method ( which looks great ), how do you determine the line ending for output. For example, if your modifying a window's file in Linux, how do you make sure you output is using the correct line ending?

      Thank you

      "Well done is better than well said." - Benjamin Franklin

[reply]

        It is my experience that most text-processing tools don't preserve newline style (file-copying tools would be a different story). I don't expect it from other tools and I generally don't write tools that do such. If I'm reading newline-terminated lines, then I'm likely processing text, not copying data.

        I expect tools to write new text files using the default newline style of the tool's platform. That is certainly what will happen with any tool written in C under Windows (unless the developer goes out of their way to change default behavior).

        A simple "perl -pe1" using native Windows perl will include "\r"s in the output even if the input lines are terminated with just "\n"s (note that I didn't even have to give perl the "-l" option).

        If you want to preserve newline style, then you want to at least binmode your input and output. Then you can decide whether you want to try to preserve inconsistent or non-standard newline style. I'm not a fan of either so I'd probably just support two output choices, either always "\n" or always "\r\n". 

        binmode STDIN;
binmode STDOUT;
my $newline;
while(  <STDIN>  ) {
    $newline ||= /\r/ ? "\r\n" : "\n";
    s/\s+$//;
    ...
    print ..., $newline;
}

        [download]

        But at this point I don't see ever wanting to do that. :)

        - tye        

[reply]
[d/l]
[select]
Re: \r\n at end of line
by Anonymous Monk on Apr 19, 2011 at 02:17 UTC
    eol
    EOL - This module aids in the conversion of text file newline characters within the context of other perl code; includes command line executable.
    PerlIO::eol - PerlIO layer for normalizing line endings
    Text::FixEOL - Canonicalizes text to a specified EOL/EOF convention, repairing any 'mixed' usages
    s/\s+$//;
[reply]
Re: \r\n at end of line
by John M. Dlugosz (Monsignor) on Apr 19, 2011 at 03:33 UTC
    So, I'm thinking that my function should insist that lines be normalized, and that the automatic endings is done with IO Layers. I'll follow chomp's example. Apparently opening a file with :crlf makes it tolerant of \r\n like I'm used to.
[reply]
Re: \r\n at end of line
by educated_foo (Vicar) on Apr 19, 2011 at 00:43 UTC
    s/\r?\n\z//

    (Does Perl remove a plain "\n" in Windows? If so, I'm surprised, since $/ isn't a regexp.)

[reply]
[d/l]
      A :crlf layer is used by default on Windows, converting CRLF to LF on read and vice-versa on write. Therefore, $/ is "\n" on Windows too.
[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom