in reply to Running out of resources while data munging

I've been processing ~200 meg files of ASCII records by reading them into a hash of array references, and then writing each @{ $hash{$key} } record into its own file.
It sounds like you are just binning the lines based on some criteria. Can you output the lines directly without first collecting them in the hash? Something like:
my %FH; while (<INPUT>) { my $key = ...determine key from $_... my $fh = $FH{$key} ||= open_fh_for_key($key); print $fh $_; } for my $fh (vals(%FH)) { close $fh }; # ... or just wait for %FH to go out of scope
If this approach would work but you would open too many files, there's work-arounds for that situation.