Unfortunately, having no file *is* a legal situation (and, from the processing logic, equivalent to an empty file)

Then change the logic, and make no file different to an empty file. That is, make it so that the create always creates a file, even if that file is empty. That way, your reader will always wait until there is a file before deciding upon its next action.

Having the absence of a file be logically equivalent to the presence of an empty file creates the situation where the reader takes the crash of the writer to mean the same thing as the writer creating an empty file. And that's just silly. It just creates a false dilemma.

With locking, the reader would have to wait for the writer to unlock the file. This will normally be a short period, but could potentially be forever if the creator crashed whilst holding the lock.

If the reader waits (with timeout) for the appearance of the file you have the same situation.

rename is atomic (and overwrites) on most common Perl platforms.

I didn't consider rename, because rename says:

... it will usually not work across file system boundaries, even though the system mv command sometimes compensates for this.

As for atomicity, I wonder how Perl could guarantee that, without assumption of the file system being used. Unfortunately I don't know enough about Samba, CIFS and NFS, to know what they offer for atomic rename. May I ask you where you got the information that rename is atomic?
-- 
Ronald Fischer <ynnor@mm.st>

NFS is notable for not supporting atomic rename. I'm sure that SMB/CIFS supports atomic rename, even over the network, but the Wikipedia pages for NFS and SMB are silent on the features of the filesystems/protocols.

Note the "file system boundaries". What that is referring to is trying to rename a file from one drive/partition to another, not watching the rename happen on a remote system.

IE: You can't rename a file from your hard disk to a floppy disk. Just doesn't make sense. Any such rename/mv operation would be done via a (very slow) copy, then a delete.

On the other hand, if you're renaming a file on the same partition, or in this case the same directory even, then all you need to do is change the name; the physical location of the file on the media doesn't need to change, and the file contents won't be partially written at any point during that operation. You simply won't see the file until the name begins matching the string you're looking for.

You are writing the file to one file system and then "renaming" it to a different file system? Why? That's rather silly. Renaming to a different file system requires that the contents be copied, which means races leading to partial content. If you need to do that, then I'd copy the file to the new file system under a temporary name (even if you use /bin/mv to do the copying) and then atomically rename from the temporary name (which the receiver should know to ignore) to the real name (which might involve two different directories on the final file system).

You haven't even bothered to read any Unix man pages for rename? Did you want an engraved invitation? They even answer your question about NFS.

Note that Perl's rename is atomic on Win32 including across Samba. No, I don't have links that say this ready to hand to you. You might have to do some research.

- tye