in reply to Re^3: Reducing Memory Usage
in thread Reducing Memory Usage

Here's a better strategy:
  1. You're grouping stuff. Ok - build a temporary file that has those fields grouped. DO THIS FIRST
  2. You're sorting stuff. Ok - take the file from 1. and sort it into another temporary file. Preferably, you would use the Unix sort command, but slurp'n'sort'n'glop works just fine, too.
  3. Now, you should be able to work with the second temporary file because it's been massaged to what you want.

There is never a rule that says you have to work with the crapola you were given! Instead of massaging in RAM, massage on the hard-drive!

------
We are the carpenters and bricklayers of the Information Age.

Then there are Damian modules.... *sigh* ... that's not about being less-lazy -- that's about being on some really good drugs -- you know, there is no spoon. - flyingmoose

I shouldn't have to say this, but any code, unless otherwise stated, is untested

In Section Seekers of Perl Wisdom