http://qs1969.pair.com?node_id=410795


in reply to Re: Ways to sequence calls from multiple processes
in thread Ways to sequence calls from multiple processes

Your program will likely work in 99% of the cases (maybe 99.9999%), but as the opening and locking are two separate steps, it might happen that:
  1. Process A opens lock.txt
  2. Process B opens lock.txt
  3. Process A flocks()
  4. Process B tries to flock().... and fails
So you should really encapsulate your code above in a polling loop that detects locking failures and retries every n seconds on failure.
Eh... what?!?!? There are several things wrong with what you say here.
  1. Opening a file will always work whether it is locked or not. Actually, the fact whether it's may be locked is irrelevant.
  2. Nobody specified use of LOCK_NB — instead he's using LOCK_EX (2), so flock will simply wait if the file is locked, until it can get a lock. No looping necessary. flock will only return after a failure if something is seriously wrong. Your program shouldn't loop then, but die.
There simply is no race condition.

If you want to be more safe, you should ask yourself: what happens if now_access_the_common_resource() crashes or loops? will the file stay locked forever? how do you detect that the other processes are starving?

Hold that thought. Well, one surely nice thing by OS-native file locking, is that a flock is automatically released if a process ends without releasing the lock itself, or closing the file. That includes crashes and being killed by other processes. There simply is no problem.

A problem could indeed be looping forever without releasing the lock... so don't do that.

My suggestion is: if your program already uses a RDBMS, why not using its own locking features? each and every decent RDBMS already has consistent locking features implemented to make its own basic work.

You could - for example - have a one-row table with a status field that each process tries to update like UPDATE mytable SET status=1 WHERE status=0 LIMIT 1. After each update you check the number of affected rows; it it's 1 you got access to the resource (and atomically locked it), if no row was affected you could not obtain the access, so you sleep() random seconds and try again. To release the resource you simply UPDATE mytable SET status = 0 WHERE table_id = 123.

That approach is simply too attrocious for words. Your solution clearly suffers of the problem you just warned against: what if the program that holds the lock, dies? The status field of that database row will simply never get cleared.

And you're actually not using "the locking features" of the database, you're using the fact that the update of the database is atomic. I'm not even 100% convinced that it is. You are using the return value of the count of affected rows for a purpose that it was never intended for.

There even are ways to try create a file if it doesn't exist, atomically. Look up O_EXCL in perlopentut. That would seem to be a far simpler, and more reliable way to do it. And even that seems to me like you're working too hard, because you still have to guard against premature death of your process, and take measures that the file will indeed get deleted in due time.

Just stick with locking.