in reply to Combining Data Files

I had a simular situation, but had to move large files from one server to another once the file was complete. What I did was use File::stat and checked the mtime, if it hadn't changed in the last 10 minutes I assumed the upload was complete.

Replies are listed 'Best First'.
Re^2: Combining Data Files
by jhourcle (Prior) on Mar 23, 2005 at 20:45 UTC

    I can't say enough about using end of file markers to ensure that the file is intact. The 10 minute wait works great when you're on a stable network, and the two machines are nearby, but when you start getting to larger files, and systems further apart, you run the risk of the transfer dying in the middle. This is especially bad if you can't back out of whatever you're about to do if you start processing.

    I've done a number of different tricks over the years. For those times when you control the format (and you can't change to SOAP, or something that's intented for this sort of thing), I make sure that all files have the last line of END (or something else that will never show up in the file)

    If you can't modify the format, and can't easily test if the file is complete, I'll send two files -- if the important file (the one you actually care about) is filename, the second file is file filename.done, with no real data in it.

[reply]
[d/l]
[select]

      That second file (filename.done) can contain a digest or checksum, if tht thing really matters. :)

      --
      brian d foy <bdfoy@cpan.org>
[reply]

In Section Seekers of Perl Wisdom