Another option which I'm surprised hasn't been mentioned yet is the use of the Tie::File module - This module, part of the standard 5.8.0 distribution, allows lines of a file to be manipulated via a tied-array without requiring the entire file to be read into memory. Additionally, the amount of memory used for read caching and write buffering can be controlled by the memory argument to the tie constructor.

Using this code, you code might look like:

#   Tie the array @file to /path/to/file using a maximum of 1Mb of mem
+ory

tie my @file, 'Tie::File', '/path/to/file', 'memory' => 1_000_000 or
    die "Cannot open file - $!";

foreach my $line ( @file )
{
    #   ... Your code follows ...
}
[download]

This module is quite stable, despite the beta label given to it by its author, and works exceptionally well in a production environment (even under 5.005.03).

 

perl -le 'print+unpack("N",pack("B32","00000000000000000000001000110010"))'