grinder has asked for the wisdom of the Perl Monks concerning the following question:

People, does anyone know of a module that does the following?

I have a program that takes a file and processes it.

Some time later, I want to run the program again on the same file, but this time, when the file is opened, the file pointer is positioned to pick up where it left off last time. The first read will then resume from the line n+1, with respect to the previous run.

Obviously this involves a trivial amount of work with seek and tell, and some way of squirrelling away the necessary context somewhere else.

There's nothing under IO::File, and searching for 'file', 'read', 'resume', 'reopen' and/or 'seek' doesn't turn anything up. Any pointers welcomed.

• another intruder with the mooring in the heart of the Perl

Replies are listed 'Best First'.
Re: Reading from where I left off
by gri6507 (Deacon) on Jan 12, 2006 at 16:34 UTC
    In the past I have done this using seek and tell and a hidden file to store the line number. It was so easy to do it that way, that I never even considered looking for a module.
[reply]
Re: Reading from where I left off
by merlyn (Sage) on Jan 12, 2006 at 16:36 UTC
    You want tell, which on most sane operating systems returns a number that represents the number of bytes you are into the file (where the next read will start). Record the number, then use an absolute seek after you've reopened the file to get back to the same place.

    -- Randal L. Schwartz, Perl hacker
    Be sure to read my standard disclaimer if this is a reply.

[reply]
[reply]
Re: Reading from where I left off
by swampyankee (Parson) on Jan 12, 2006 at 17:13 UTC

    I've not tried anything like this, but you could try using the Storable module. What I would do (you can probably come up with a better scheme) is something like this:

    #!perl

use strict;
use warnings;

use Storable qw(store retrieve freeze thaw dclone);

$file = shift(@ARGV);

open(FH,$file) or die "could not open $file because $!\n";

if($file_pointers = retrieve('fileposition'))
   {
    if(exists($file_pointers -> {$file})
      {
       seek(FH, $file_pointers -> {$file},0) or die "could not seek to
+ '. $file_pointers -> {$file} . ' because $!\n";
      }
   }

while(my $keepreading) 
{
   if(defined($line = <FH>)
   {
          process($line);
          $keepreading = continue_question();

          $file_pointers -> {$file} = tell(FH);
   }
   else
   {
          delete $file_pointers -> {$file};
   }
close(FH);

store $file_pointers, 'fileposition');

    [download]

    Note that this code is NOT tested nor even compiled.

    emc

    " When in doubt, use brute force." — Ken Thompson
[reply]
[d/l]
Re: Reading from where I left off (tmi)
by tye (Sage) on Jan 13, 2006 at 08:06 UTC

    I did this but that ISP went out of business /very/ suddenly so I doubt I have the code now.

    I stored the inode of the file along with the seek position as the log I was incrementally parsing would be rolled once each month (you may also want to start from the front of the file if your saved seek position is beyond the end of the file).

    One trick was that you likely need to handle the last line not containing a newline, which means you shouldn't process it yet. I did tell before each 'readline' and retuned if no newline got read, leaving me with the start of that line as the last place I telled.

    - tye        

[reply]

Back to Seekers of Perl Wisdom