in reply to Re: Parsing a text file
in thread Parsing a text file

graff, be careful about \r and \n. I changed my habit of writing \n when I'm not sure where my script will be run. The reason is this section from perldc perlipc section Internet Line Terminators

Internet Line Terminators

The Internet line terminator is "\015\012". Under ASCII variants of Unix, that could usually be written as "\r\n", but under other systems, "\r\n" might at times be "\015\015\012", "\012\012\015", or something completely different. The standards specify writing "\015\012" to be conformant (be strict in what you provide), but they also recommend accepting a lone "\012" on input (but be lenient in what you require). We haven't always been very good about that in the code in this man- page, but unless you're on a Mac, you'll probably be ok.

in this case, where the file is edited by several people on different platforms it might be a good idea to use a combination of \012 and \015

s$$([},&%#}/&/]+}%&{})*;#$&&s&&$^X.($'^"%]=\&(|?*{%
+.+=%;.#_}\&"^"-+%*).}%:##%}={~=~:.")&e&&s""`$''`"e

Replies are listed 'Best First'.
Re^3: Parsing a text file
by ikegami (Patriarch) on Jan 14, 2009 at 05:20 UTC

    That's part wrong, part outdated.

    • In the situation where "\r\n" ends up being "\015\015\012" (when using :crlf), so will "\015\012".
    • "\012\012\015" should read "\012\015", and that only occurs on MacPerl (Perl for Macs earlier than OS X).

    On all current operating systems, "\015" is interchangeable with "\r" and "\012" is interchangeable with "\n".

    (Well, not on EBCDIC systems. But it's not clear to me how you'd want the program to behave there in this case.)

[reply]
[d/l]
[select]
Re^3: Parsing a text file
by Gangabass (Vicar) on Jan 14, 2009 at 05:40 UTC

    But if you look a bit more into graff code you'll see that he's already use a combination

     /[\r\n]+/

    [download]
[reply]
[d/l]

      [\r\n] does appear to finesse the problem nicely.

      Unfortunately, when this last came up, I looked at all the relevant documentation I could find, but I did not see any guarantee that "\r" will be "\x0A" if "\n" is "\x0D" (or vice versa) in not-EBCDIC land. Or even that "\r" and "\n" are in general guaranteed to be duals of each other.

      As brother ikegami says, you know and I know that these days, with the exception of EBCDIC systems, "\r\n" is exactly "\x0D\x0A". If an authoritative position were taken that as of (say) 5.8.0:

      • "\r\n" eq "\x0D\x0A" except for EBCDIC.

      • any system using line endings other than "\n" will support, and will by default use, a PerlIO layer than maps those line endings to/from "\n"

      then we could consign worrying about this piece of magic to the bin. I don't know what the position is with MacPerl, but perlmacos suggests that the above could be back-dated to 5.8.0 including MacPerl.

      FWIW, socket handling can (of course) be simplified by applying binmode $sock, ':crlf', which is nice. Nevertheless, chomp is a snare and a delusion if you think it's handling Internet CRLF line endings (unless you're futzing about with $/ at the same time). Wouldn't it be nice to have a chompnl equivalent to s/\x0D?\x0A$// ? And, perhaps, chomps equivalent to s/\s+$// ?

      BTW, I note that \R is defined in perlreref as (?>\v|\x0D\x0A). Shouldn't that be (?>\x0D\x0A|\v) ? And I wonder what the EBCDIC folk make of this !

[reply]
[d/l]
[select]
        You (and Skeeve) seem to be confused about the difference between the use of "\r" and "\n" in file I/O and their use in regular expressions. To my knowledge, there has never been any magic, variation or ambiguity when using these in regular expressions -- things like s/\r/\n/g; and tr/\r//d; and split /[\r\n]+/ have always had clear and consistent meanings and effects, as documented near the top of the perlre man page.

        But of course some hapless programmers, faced with data they don't fully understand, have been (and continue to be) confounded by these clear and consistent expressions when they use the wrong ones on a given set of data. In other words, it's not the expressions themselves that are problematic, it's the misunderstandings about what is in the data, which stem in part from not understanding the source(s) of the data and/or not using a suitable method to read it in (or write it out).

        The various wrinkles of file I/O magic, including what "chomp" does, were relatively simple in pre-5.8 perl (though still tricky enough to burn the unwary in numerous ways); the situation and methods of control in 5.8 and later versions are more varied and intricate, and it's a tribute to the designers of PerlIO that the older, simpler idioms still do what they always did.

        Update: As for this point:

        BTW, I note that \R is defined in perlreref as (?>\v|\x0D\x0A). Shouldn't that be (?>\x0D\x0A|\v) ? And I wonder what the EBCDIC folk make of this !

        I see that "\R" and "\v" were both introduced as of 5.10, and these are likely to help once people realize they exist and get the hang of using them. Maybe there's a problem with the description in that man page, but their actual behavior looks very handy: 

        $_ = "hi\x0bthere\x0d\x0aline 3?\x0a    line4\x0dno\x0away\r\n";
print;

@lines = split /\R/;
print "===\n", join( "\n===\n", @lines ), "\n===\n";

        [download]
        For me, that snippet produces: 
        hi
  there
line 3?
no  line4
way
===
hi
===
there
===
line 3?
===
    line4
===
no
===
way
===

        [download]
        Notice how vertical tab (\x0b) and the isolated CR (\x0a) are treated the same as CRLF and LF. Wow, this is going to make a lot of things easier.

        Another update: And I wonder, just who are all these EBCDIC people I keep hearing about? Are they in the same museum with the VAX/VMS users?

[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom