in reply to Re: flock() broken under FreeBSD?
in thread flock() broken under FreeBSD?

Following the debugging technique of "imagine how something might happen, then go confirm it," here's a story for how the behavior tye observes might happen. It involves an imagined implemententation of FreeBSD flock(2), and might provide some guidance for someone who cares to dig into the FreeBSD source.

Assume an OS implementation of flock() that either intentionally or inadvertantly gives priority non-blocking requests. That is, a non-blocking flock() request will be satisfied without unblocking other processes that are waiting to aquire a lock, even though the non-blocking request releases its prior lock first. (Ignore whether this is sensible, and just assume that it's coded that way.)

Now consider this scenerio: Process A holds a shared lock on F. Process B blocks on a blocking requests to acquire an exclusive lock. Process A makes a non-blocking request to "upgrade" its lock to exclusive. Now, according to the flock(2) man page, this means releasing the shared lock first. But, since the request is a non-blocking one, and since the flock() routine is coded to give priority to non-blocking requests, process A acquires an exclusive lock, even though B was waiting first. B is still blocked. Following the same logic, A can then repetitively "downgrade" the lock to shared, and upgrade to exclusive, all without unblocking B. B is starved until either A makes a blocking flock() request, or A releases the lock by an explicit close or by process termination.

This is how it might happen, given the code tye provides. Can someone with access to FreeBSD sources (and the will to use them) confirm whether this is what's going on?

Replies are listed 'Best First'.
(tye)Re: flock() broken under FreeBSD?
by tye (Sage) on Jul 28, 2002 at 05:43 UTC

    Well I tried my code on Linux and the failure case looks a little different: 

    $ ./locktest & sleep 4; ./locktest
[1] 1553
Using flock()...
1553 shares.
1553 owns
1553 shares
Using flock()...
1557 shares.
1557 waiting for previous instance(s) to exit...
1553 owns
1557 owns
Running...
1557 owns
1553 can't revert self lock to shared: Resource temporarily unavailabl
+e
1557 shares
^C
[1]+  Exit 11                 ./locktest

    [download]
    Which demonstrates that Linux doesn't have the strong preference for non-blocking requests like FreeBSD appears to have.

    Having lock up-/down-grading introduce a race condition where the lock is freed first is such a horrid design choice to my mind that I didn't even consider the possibility when reading "man flock" (this is not even mentioned in Linux's extremely short version of "man flock" tho my test cases show that it is the case there as well).

    Thanks for the enlightenment. Now I have one more reason to hate flock. I should find a module that provides a convenient wrapper for fcntl locks... (:

            - tye (but my friends call me "Tye")
[reply]
[d/l]
      The FreeBSD man pages hint at this behavior:

      A shared lock may be upgraded to an exclusive lock, and vice versa, sim­ply by specifying the appropriate lock type; this results in the previous lock being released and the new lock applied (possibly after other pro­cesses have gained and released the lock).

      Requesting a lock on an object that is already locked normally causes the caller to be blocked until the lock may be acquired. If LOCK_NB is included in operation, then this will not happen; instead the call will fail and the error EWOULDBLOCK will be returned.

      So, if you try to upgrade your shared lock, you release the lock and get in line behind everyone else blocked for an exclusive lock. If you try to upgrade your exlusive lock to a shared lock, you release the exclusive lock and get in line (behind everyone blocking for an exclusive lock) for a shared lock.
[reply]
      What would you want to happen if 2 processes get read-only locks and then try to "upgrade" them at the same time to read-write? What reasonable semantics can you suggest for resolving this?

      If there is no reasonable way to implement upgrades, then it is not unreasonable to say, Silly programmer. Upgrades don't exist. If you will want to write, then you should tell me that from the start!

      Getting a shared lock is a promise that you won't be doing any modifications based on your read. Don't make that promise if it isn't true.

[reply]

        What would you want to happen if 2 processes get read-only locks and then try to "upgrade" them at the same time to read-write? What reasonable semantics can you suggest for resolving this?

        If they both request blocking upgrades then it is called "deadlock". It is something you have to deal with (at least to the point of planning how to avoid) in nearly any situation dealing with locking. Most of the systems I work with these days have a very simple response to deadlock, for example, "man fcntl" says "This implementation detects that sleeping until a locked region is unlocked would cause a deadlock and fails with an EDEADLK error." And it is easy to create deadlock with flock so avoiding a case of deadlock is not a reasonable explanation of this particular design choice. Especially since non-blocking upgrades would be very useful and can't be a source of deadlock.

        Note that this case wouldn't have been one involving deadlock. The reasonable semantics I wanted were exactly those provided by fcntl-based file locks. I can create the deadlock you describe with my test program by having two instance try to start at nearly the same time. And fcntl handles this very nicely and the second one to try to do a blocking upgrade to an exclusive lock simple fails with "Resource deadlock avoided" (but my original test code doesn't test for this failure).

        Having an operation involving locks be implemented non-atomically is, frankly, just stupid to me. But you only have to try to make practical use of any number of parts of Sys V IPC to see that stupid designs do become standard. For example, "man fcntl" says:

        This interface follows the completely stupid semantics of System V and IEEE Std1003.1-1988 (``POSIX'') that require that all locks associated with a file for a given process are removed when any file descriptor for that file is closed by that process.
        Which is a pretty stupid idea in a lot of situations. I can see how you can look at it from a different angle and think of it as a feature but when you look at both sides, the trade-off is pretty clear and this "feature" would certainly be changed if it weren't for backward compatibility augmented by standardization.

        Getting a shared lock is a promise that you won't be doing any modifications based on your read.

        I'm not sure where you got that idea. To me, a shared lock means that I don't want anyone else to change the data out from under me. It is quite reasonable to then decide that I need to make a change to the data and request access to change it. If I can't request that without first releasing the shared lock (in a non-atomic manner), then I'm forced to hold the exclusive lock while I reread the data and reperform the computations to decide whether I still want to make the same change. This can be very wasteful.

                - tye (but my friends call me "Tye")
[reply]

In Section Meditations