Take a look at the sliding window technique. It helped me with a job that required line-by-line tests. It basically works by batching the disk I/O.