This is the fourth in a series of questions relating to what appears to be the same project. Maybe it is time to stand back a little and describe your overall project rather than have us continually trying to guess what you are trying to achieve and squeezing information out of you a single small drop at a time?

So far we have some information concerning the format of a couple of input files. We know that at least one of these is big. We know that you are selecting some data based on some other data. We know there is a third file involved.

We don't know what you are trying to achieve in a "big picture" way. We don't know if this is a one off. If this is not a one off we don't know how the input data changes over time. We don't know if you need to perform multiple searches with the same data.

You seem to focus on answering a few of the questions you've been asked and you seem to be looking for a quick fix solution to what is probably a small part of the problem. The more we know about the high level problem the more we can offer ways to address the big issues.