in reply to When file is committed to disk?

I'm not sure I understand the question. It sounds like you already know how many files should be in the directory in order for it to be "done being written", and you seem to also know the intended "full size" of the files. Where does this information come from? Where do the files come from?

How are you able to tell that -s or stat() on a large file "just reports the full size before it's all there"?

Are you able to configure or monitor the source (or the process) that is putting files into the directory? If so, you could set it up to issue some unambiguous event when it's done (e.g. create a special little file), or just check whether it's still running.

If you don't have that sort of access to the source, well, it depends. If the receiving directory were on a unix/linux machine, the -s or stat() would report how much data has been stored so far, and your loop approach would work as expected. Oh well.

Isn't it the case that windows prohibits certain kinds of access to a file when some process is writing to that file? If that happens to be true in your case, maybe you can figure out some harmless operation that will fail while the file is being written, succeed only when the writing process has finished and closed the file, and in any case does not change the file content. Good luck with that.

Replies are listed 'Best First'.
Re^2: When file is committed to disk?
by Anonymous Monk on Jun 19, 2009 at 06:28 UTC
    Recently I had problem with DBM-Deep-1.0014/t/53_misc_transactions.t, it was eating all my HD space. My laptop is slow, it takes maybe 5 minutes to write 3-4gb, and during that 5 min the full file size is reported. Also you can't kill the perl process until write buffer is flush, it still shows with ps.