suneel.reddy:

I helped someone with a similar problem. I suggested that they sort the files on the key of interest, and then process the file sequentially.

...roboticus

When your only tool is a hammer, all problems look like your thumb.