Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hello monks, hope you can help with this one... I've got a file which at the minute read in the way of using Storable::retrieve, but it's getting too big to read all into a hash at once. How could I read it for instance 10 lines at a time? Cheers, Tommy

Replies are listed 'Best First'.
Re: I've got an enormous...file
by dragonchild (Archbishop) on Apr 09, 2008 at 14:09 UTC
    DB_File is good. DBM::Deep was designed for this. And, the nice part is that you don't need to change any part of your script other than the Storable::retrieve part. The rest just assumes that it's a hash. :-)

    My criteria for good software:
    1. Does it work?
    2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?
Re: I've got an enormous...file
by citromatik (Curate) on Apr 09, 2008 at 14:04 UTC

    If I understood correctly you are looking for a way of reading a file in chunks of 10 lines. Probably you can use cpan modules for that (Tie::File or DB_File). But this is another way (using an iterator):

    use strict; use warnings; sub getNlines { my ($fh,$nLines) = @_; return sub { my $linesRead = 0; my $rec; $rec .= <$fh> for (1..$nLines); return $rec; } } open my $ff, "<", shift @ARGV or die $!; my $get10lines = getNlines($ff,10); ## will retrieve 10 lines per cal +l while (my $rec = $get10lines->()){ ## Process next 10 lines print "$rec\n"; }

    Hope that this helps

    citromatik

Re: I've got an enormous...file
by Anonymous Monk on Apr 09, 2008 at 13:18 UTC
    Storable is all or nothing, use DB_File

    mine is bigger

Re: I've got an enormous...file
by CountZero (Bishop) on Apr 09, 2008 at 16:24 UTC
    Just to be curious, what is the size of your file and as from what kind of size you started to have problems?

    CountZero

    A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James