Thanks for the advice. I am currently going over the docs.

Out of curiosity, how did you resolve the atomicity issue? I mean, did you use some other Perl technique or did you move that part out of Perl altogether and used something else?

What do you mean by flock being graceful? You mean if flock somehow crashed or misbehaved it didn't mess up things?

Thanks again!

I just realized I never answered your question about what I meant by "graceful".

You ask Perl to get a lock on the file, and off you go. Cleanup is a matter of closing the file.

No funky coding, no oddball exception handling, no having to overthink the problem. Ask it to do its job, and it does -- no muss, no fuss, no thirty lines of code. That, to me, is the epitome of grace.

If memory serves, I coded up a primitive two-stage, file-based, self-help locking mechanism. Something akin to what follows. In this example, for file test.dat:

1. Create a file (lock request token) based on the original filename
   • Include its creation timestamp
   • Include its modification timestamp
   • Include its server and unique PID
   • Make these sortable by creation, server, pid, modification
2. Loop until it is the oldest lock request for this file
   • Re-create your lock request token file if it's been removed.
   • Update your lock request token modification timestamp
   • Delete requests older than the lock request timeout
   • sleep a short bit (don't burn the CPU)
3. Create another file (lock bid token) based on the original filename
   • Include its creation timestamp
   • Include its modification timestamp
   • Include its server and unique PID
   • Make these sortable by creation, server, pid, modification
4. Loop until it is the oldest lock bid for this file
   • Re-create your lock bid token file if it's been removed.
   • Update your lock bid token modification timestamp
   • Delete bids older than the lock bid timeout
   • sleep a short bit (don't burn the CPU)
5. Unlocking consists of removing the lock request and lock bid files
6. Long-running processes need to have a mechanism for updating the modification timestamp on the bid token to avoid it being deleted by others using the same self-help system.

Then requests for lock simply queue up. This is not collision-proof, but in a low-volume environment was sufficiently collision-resistent for my needs (the math was favorable).

Wow. I discourage anybody from trying to repeat this exercise.

This looks like an example of something I've seen happen many times: A relatively inexperienced (with locking, anyway) person tries to solve a problem with locking by piling on complexity. The likely best solution (IME) is usually actually removing complexity. Part of why I know of these things is because I've had to step in and fix the problems that resulted when the implementation grew beyond the "toy" stage.

I'd love to hear some actual facts or details about the purported problems with flock. The description makes me suspicious in itself because Perl's flock is just a thin wrapper around exactly one function from your operating system; either flock(2), fcntl locks, lockf (on systems I appear to never have used), or LockFileEx (on Windows). So I'd expect any discussion of problems with Perl's flock to begin with specifying which of those choices the problem applies to. If you don't start with determining that, then you are doing some stabbing in the dark.

The primary problem I can think of with Perl's flock is that it is most often implemented via flock(2) and that doesn't work over NFS. That's one reason that I wish Perl would prefer fcntl locks over flock(2). But using fcntl locks directly from Perl is not particularly hard so I'll often do that (even when not dealing with NFS; fcntl locks are just much more powerful and also have slightly better semantics at the edges than flock(2), IMO).

Now, it is rare that I find myself on a system where fcntl locks don't work over NFS. I find lots of complaints about that, but I've just almost never run into a system with such problems. These days, you really should just expect that your Linux supports fcntl locks over NFS unless you are using something fairly ancient.

But even if you've got a system where fcntl locks don't work over NFS, the answer is not something complicated. Just use link. Create a file named "lock.$$.$host" and write your host name (or IP address) and process ID into it. Then call link("lock.$$.$host","lock"). If that works, then you own the lock (and you can then unlink "lock.$$.$host"). If it fails, then somebody beat you to it.

But I almost never do that because I sometimes want to reliably detect if the process holding the lock is actually still running. But then, NFS rather sucks anyway (failure is handled badly and the "fix" for that, autofs, actually means failure is handled even worse, at least in some respects) so I prefer to just avoid it anyway.

And if you aren't dealing with both NFS and an ancient OS, then you can also just pass O_EXCL to sysopen. That calls open(2) and that manual's section on O_EXCL even documents how to use link to get reliable locks over NFS simply (just like I described above).

So, you can use Perl's flock, fcntl, sysopen, or link, roughly in that order and each of those is simpler, more reliable, and faster than the proposed solution I replied to.

- tye