in reply to Best way to manage memory when processing a large file?

As to the efficiency of your data structure in memory, I'll leave that to others. Being as I forget from time to time and am stung by the lack of defined in the below, here is a basic while loop for reading line by line.

The defined call ensures your loop does not break prematurely should the line evaluate to a false value.

use autodie; open(my $fh, '<', $file); while ( defined (my $line = <$fh>) ) { chomp($line); # do stuff }