Re^2: commands with multiple pipes in perl

Hi, but the approach you have mentioned here takes a long time as it searches extensively. when i run the entire command from the command line it gives me the output in seconds but when i put it in the script it takes a long time and i had to end the script.even close to 5minutes and no output.. I also reduced the file size and tried still takes a long time making me wonder if the script has hung

Comment on Re^2: commands with multiple pipes in perl

Replies are listed 'Best First'.
Re^3: commands with multiple pipes in perl by ELISHEVA (Prior) on Mar 17, 2009 at 12:34 UTC
Curious. It shouldn't be doing any more searching than grep would, assuming you are reading in one line at a time. (if you slurped the file in as one long line that could slow you down a lot). How did you adapt the above code for your situation? Perhaps if you posted the code we might have a better idea of why your program is so slow. When I created a dummy file with the data above repeated 10,000 times (equivalent to a 6.6M file) parsing took only 0.71 seconds (wall clock time). When I upped the size by repeating the file 1,000,000 times (equivalent to a 660M file, more than half a gigabyte) it took 26 seconds. Best, beth	[reply]

Replies are listed 'Best First'.

Re^3: commands with multiple pipes in perl
by ELISHEVA (Prior) on Mar 17, 2009 at 12:34 UTC

Curious. It shouldn't be doing any more searching than grep would, assuming you are reading in one line at a time. (if you slurped the file in as one long line that could slow you down a lot). How did you adapt the above code for your situation? Perhaps if you posted the code we might have a better idea of why your program is so slow.

When I created a dummy file with the data above repeated 10,000 times (equivalent to a 6.6M file) parsing took only 0.71 seconds (wall clock time). When I upped the size by repeating the file 1,000,000 times (equivalent to a 660M file, more than half a gigabyte) it took 26 seconds.

Best, beth

[reply]