in reply to Re^4: Making this script process 56,000 lines 5 times faster
in thread Making this script process 56,000 lines 5 times faster

Actually your original approach in the OP with for( split ...) had this problem.

The while(m/.../) here is an iterator, you could operate with a sliding window to read consecutive chunks into memory, and adjust with pos where you left of.

Cheers Rolf
(addicted to the Perl Programming Language and ☆☆☆☆ :)
Wikisyntax for the Monastery

Replies are listed 'Best First'.
Re^6: Making this script process 56,000 lines 5 times faster
by kris004 (Initiate) on Mar 22, 2018 at 19:03 UTC

    Ah ok, thanks!