I can't believe how many responses say to just add more memory. I realize this might be tongue-in-cheek, but the responsible thing for a developer to do is to figure out how to consume less memory.

We haven't seen any code yet, nor any sample input. Thus, the question is mostly unanswerable. But attempts at providing suggestions for an unanswerable question ought to at least lean in the direction of what a developer might do to solve the problem, rather than what a hardware tech might do.

If at all possible, read your input one record at a time, process that record, write out anything that needs to be written, accumulate whatever needs to be accumulated, and then discard from memory the current record and move on to the next. Some variation on this theme is how most of these scalability issues are resolved. Maybe you need to hold onto two or three records. Or maybe you can immediately drop any records that are uninteresting and only hold onto ones that need further work. But in the end your approach will probably look something like this.

Another technique might be to break up the list of records into smaller chunks, and work on them individually. Accumulate results for each chunk, and in the end, compile results for all the accumulated data. This is a "map-reduce" strategy.

So these are some very generalized suggestions, which are about as specific as possible given the ambiguities of the original question.

To the original poster: Please consider this: When you present a question that is devoid of detail, it usually means that you have become so exacerbated with the problem that you've given up trying to figure it out yourself. But we have no chance of figuring it out either unless presented with those very details that have so frustrated you.