in reply to reversing files line by line

The ways to reverse lines get a bit messy because they involve merging segments together (provided of course as your problem statement says that you can't fit all lines into memory at once). As it turns out well written system sort routines handle huge files and all this merging can split the job up into smaller pieces.

My suggestion would be to make a pass thorough this huge file, adding a number in the first column while generating a new file. Then use a system sort on that file using this first column. Then make a pass through the sorted file to remove this "index" number that was added in the first pass. And you have a reversed file.

This idea may sound "stupid", but you may be surprised at how fast this can work. It is dependent upon a lot of things. But you can write the code in some number of minutes and let it run while you are pondering more efficient ways. And of course there are more efficient ways!

I don't want to get too involved in the more complex ways until I hear back about how fast and what limitations were involved with this "stupid idea".

I just propose that you write some simple code and launch it off on its job, while thinking about other ways.

Update: This "tac" idea is also good. It may or may not be faster, depending upon how it is implemented. The main point is try the simple things first.