EyeOpener has asked for the wisdom of the Perl Monks concerning the following question:

Hello wise ones:

I've got a script which monitors a directory on an AIX system. If a new file is added, it opens the file, munges it around, and moves the original. Here is what the meaty part of the script looks like: 

while (1) {
    my @files = glob "*";
    foreach my $file (@files) {
        dostuff($file);
    }
}

sub dostuff {
    my $victim = shift;
    open IN, "$victim" or die "Can't open input file $victim: $!\n";
    # more stuff here
    close IN;
    rename "$victim", "/z/saved/$victim";
}

[download]

The problem is that the filename is grabbed before the file is completely written. If somewhat large files are FTP'd into this directory, the script tries to open the file while the FTP is in progress, and the script dies on the open IN line with the error "Cannot open or remove a file containing a running program."

Is there a file test I can use to determine whether the file is still being written, or is open by another process? Alternatively, it would suffice to trap the error and retry until the file is ready. I'm not really familiar with eval, and I can't find a good example of this kind of loop. Can anyone suggest a good approach? Thanks!

Peter

Replies are listed 'Best First'.
Re: Determine when file is done being written?
by Weasel (Sexton) on Jun 12, 2002 at 18:18 UTC
    trapping an error is easy: 
    eval {
  # your code to be trapped
  # ... or die "error message 123"
}
if ($@) {
  # do your error checking, such as $@=~/error message 123/
}

    [download]
    As was announced perl6 will change "eval{BLOCK}" to "try{BLOCK}" when it is used for trapping an error, to be more consistent.

    It is interesting though to have a possibility to check whether writting is currently performing.

    I think it is somewhere in IO::* modules (may be in IO::Handle), but unfortunately can not help with this right now.

    Have a nice day,
    Weasel.

[reply]
[d/l]
[select]
      Thanks Weasel and everyone for the suggestions. The eval in a bare block turned out to be a nice, simple solution, and lets me catch more than just the possible error I know about: 

      sub dostuff {
    my $victim = shift;
    {
        eval {
            open IN, "$victim" or die "Can't open the file: $!\n";
        };
        last unless $@;
        if ($@ =~ /containing a running program/) {
            print "  Waiting...\n";
            sleep 5;
            redo;
        } else {
            die "Unknown error opening file: $@";
        }
    }
    # more stuff here
    close IN;
    rename "$victim", "/z/saved/$victim";
}

      [download]

      I'll have to get comfy with eval; I can see how it could be indispensable. Thanks again!

      Peter

[reply]
[d/l]
        Let me add little comment to save you from possible problem.

        Actually eval("string") would be much slower than eval{BLOCK} because in first case Perl will parse it every time statement is executed, while second form will be compiled once, and actually it is a replacement for "try".

        Best wishes,
        Weasel.

[reply]
Re: Determine when file is done being written?
by jwest (Friar) on Jun 12, 2002 at 18:44 UTC
    One alternative approach is to watch the log of the FTP server to see if the transfer has completed. This would require clever use of File::Tail and possibly the Net::FTPServer::XferLog modules.

    The net result of this would be an event-driven methodology (dostuff() to a file when it is transfered in) as opposed to the polling methodology that you're using now.

    Not that there's anything wrong with polling. As I say, it's just an alternative. Frankly, I'd be more inclined to do as you are, with the modifications suggested by Weasle.

    Hope this helps!

    --jwest


    -><- -><- -><- -><- -><-
All things are Perfect
    To every last Flaw
    And bound in accord
         With Eris's Law
 - HBT; The Book of Advice, 1:7
[reply]
Re: Determine when file is done being written?
by how do i know if the string is regular expression (Initiate) on Jun 12, 2002 at 18:43 UTC
[reply]
Re: Determine when file is done being written?
by cmilfo (Hermit) on Jun 12, 2002 at 19:39 UTC
    Well, be it that you are on Unix, you could use the fuser command. fuser returns the pid of any process that has an open file handle to a file. There is a module on CPAN (Linux::Fuser). As the name states, however, it is only for Linux. You could use the command line fuser via backticks and capture the output. Or, if you are the adventurous type you could port the module.

    Cheers!
    Casey
[reply]
Re: Determine when file is done being written?
by Anonymous Monk on Jun 12, 2002 at 19:00 UTC
    You could keep checking the size of the file using stat, and after it has not changed for a while, assume that it is safe to do what you have to do.
[reply]

Back to Seekers of Perl Wisdom