dirac:

The strategy we use is to create the tar file with a temporary name (e.g. with an extension of .tmptgz). Then when the tar file is completed, we rename it into the final name. Processes that consume the tar file look only for files whose name matches the final name template.

Occasionally, we'll have to work with processes that don't use that strategy. In those cases you can either wait for the last modified date to be old enough that "it should be complete now", or you could dredge through lsof output to see if anyone has the file open.

...roboticus

When your only tool is a hammer, all problems look like your thumb.

I think I would do something like this:

while( my $file = getNextFilename() ) {
    my $size = -s $file;
    sleep 1;
    next if -s( $file ) != $size; ## Pick it up next time around.
    ## process complete files
}
[download]

You could try some advisory locking, flock or a lockfile, but the canonical solution is to rename a temp. file once it is complete.

The other process might create the archive in its build directory, then move it to queue directory. Unpacker process would periodically process the queue.