Re: Parse string greater than 2GB

In a one-liner:

$ perl -pe 's/\n//' /path/to/data
[download]

Of course this does mean you don't get the length shown. But that is an easy fix, simply pipe it into wc -.

~Thomas~ 

"Excuse me for butting in, but I'm interrupt-driven..."

Comment on Re: Parse string greater than 2GB Select or Download Code

Replies are listed 'Best First'.
Re^2: Parse string greater than 2GB by rjt (Curate) on Jun 30, 2013 at 09:32 UTC
You wrote: `$ perl -pe 's/\n//' /path/to/data` Your approach reads the data file (via standard input) one line at a time (delimited by newlines), and searches every line in its entirety to replace one newline character before the implicit `-p` loop prints them out. One can accomplish the same thing in about half the CPU time (depending on average line length) with: `$ perl -pe chomp /path/to/data` The OP also indicated that they have to stick with the `read()` loop, so it's worth noting that solutions like these that read line by line don't fit the problem description. (Not that I don't have some significant doubts about the problem description...)	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re^2: Parse string greater than 2GB
by rjt (Curate) on Jun 30, 2013 at 09:32 UTC

You wrote:

$ perl -pe 's/\n//' /path/to/data

Your approach reads the data file (via standard input) one line at a time (delimited by newlines), and searches every line in its entirety to replace one newline character before the implicit -p loop prints them out. One can accomplish the same thing in about half the CPU time (depending on average line length) with:

$ perl -pe chomp /path/to/data

The OP also indicated that they have to stick with the read() loop, so it's worth noting that solutions like these that read line by line don't fit the problem description. (Not that I don't have some significant doubts about the problem description...)

[reply]
[d/l]
[select]