Oh, sheesh, thargas -- your post made me realize I'd missed something basic in the original post. The first file he opens isn't the list of files -- it's the list of strings.

On a gut I'd say he's buffering a Cartesian Product of lines per file x lines in REFFILE. Can't prove it without the actual source code -- but it sure would fit the memory consumption pattern being presented.

This only enhances what everyone has been saying -- post the actual code, not this mock-up of it -- there's something structurally wrong and we'll need to see the steel to find the rust.