in reply to Thrashing on very large lines

Open the file binary mode. Read it a modest size block at a time (say 8K) and concatenate the new block onto the end of a buffer. Use index to find the line ends and pull the lines out of the buffer using substr. Top up the buffer when there are no more line ends.


DWIM is Perl's answer to Gödel