Re^2: Making this script process 56,000 lines 5 times faster

Replies are listed 'Best First'.
Re^3: Making this script process 56,000 lines 5 times faster by LanX (Saint) on Mar 22, 2018 at 18:49 UTC
> as long as I'm not worried about running out of memory. Where do you see a memory problem here? Cheers Rolf _{(addicted to the Perl Programming Language and ☆☆☆☆ :) Wikisyntax for the Monastery}	[reply]
Re^4: Making this script process 56,000 lines 5 times faster by kris004 (Initiate) on Mar 22, 2018 at 18:53 UTC
If I say, read in a 4gb file. Not really a problem with this hosts file. I assumed that if it wasn't line by line, then the whole file would have to be loaded into memory.	[reply]
Re^5: Making this script process 56,000 lines 5 times faster by LanX (Saint) on Mar 22, 2018 at 18:58 UTC
Actually your original approach in the OP with `for( split ...)` had this problem. The `while(m/.../)` here is an iterator, you could operate with a sliding window to read consecutive chunks into memory, and adjust with `pos` where you left of. Cheers Rolf _{(addicted to the Perl Programming Language and ☆☆☆☆ :) Wikisyntax for the Monastery}	[reply] [d/l] [select]
Re^6: Making this script process 56,000 lines 5 times faster by kris004 (Initiate) on Mar 22, 2018 at 19:03 UTC