Is this an OS thing

No. It shows the current size of the file, whether it's been committed to disk or not.

The writer is probably buffering its output.

>perl -MIO::Handle -MFile::stat -le"open $fh, '>tmp'; print $fh 'foo';
+ for (1..2) { print stat('tmp')->size; $fh->flush }"
0
5
[download]

is there any way for me to determine if the file is completely there or not

Sorry, it's impossible to predict whether someone will write to a file.

A common practice is to have the file creator build the file in one directory, then rename it into a different directory. The reader simply checks for the presence of files in the later directory.

How are you able to tell that -s or stat() on a large file "just reports the full size before it's all there"?

Are you able to configure or monitor the source (or the process) that is putting files into the directory? If so, you could set it up to issue some unambiguous event when it's done (e.g. create a special little file), or just check whether it's still running.

If you don't have that sort of access to the source, well, it depends. If the receiving directory were on a unix/linux machine, the -s or stat() would report how much data has been stored so far, and your loop approach would work as expected. Oh well.

Isn't it the case that windows prohibits certain kinds of access to a file when some process is writing to that file? If that happens to be true in your case, maybe you can figure out some harmless operation that will fail while the file is being written, succeed only when the writing process has finished and closed the file, and in any case does not change the file content. Good luck with that.

Recently I had problem with DBM-Deep-1.0014/t/53_misc_transactions.t, it was eating all my HD space. My laptop is slow, it takes maybe 5 minutes to write 3-4gb, and during that 5 min the full file size is reported. Also you can't kill the perl process until write buffer is flush, it still shows with ps.