This is an excellent point, and it is easier to get in trouble than you might think looking at the gap in the code.

Control of it is held at the OS level. If someone else has a flock waiting on your file, as soon as you drop your flock and lose your time slice, they get the flock and go. However the OS lies to you. You can have written output, and flushed buffers, and all that means is that the OS knows what it still has to do for you, but it has not necessarily actually happened yet. So if you do your write, drop your lock, they get control, open, and read, your write may not be visible to them.

Now you say that this is unbelievably unlikely. And it doesn't happen that often. But if you have critical data, then you probably have little tolerance for the possibility. Besides which while it is unlikely that the flock will be when you lose your time slice on a single-processor system, with SMP it is far more likely to happen. Why? Because before the kernel can drop your lock it has to make sure that all of the processors are aware of the change, and that means that your flock causes a wait on what other processors are doing. Because of that wait, when your flock is done it is not at all unlikely that you have hit the end of your time-slice. Which means that it is not at all unlikely (on SMP machines) that you will be interrupted between the flock and the close. And if that happens while someone else is waiting on the lock and they get scheduled next, their time slice can easily be long enough to grab a chunk of the file...

So as c-era correctly says, the right way to do this is just call close with the lock still held. Then the OS will know to flush all internal buffers before, dropping the filehandle, and in cleanup will drop the lock safely. And done this way there is no possible miscoordination of buffers to mess you up.

There is a moral and general principle here. Whenever you have an opportunity (not just with locks) to push the responsibility for synchronizing stuff down to a layer that thinks it is in control, do so. Doing that means less work for you, less opportunity for you to make a mistake, less possibility that the lower layer will make a bad assumption, etc.

This applies to file locking (push it off to the OS), writing loops (push it off to Perl), installing software (push it off to your package manager - at least this is good advice with Debian), and so on.

This is mostly a question. My impression is that flushing your buffers doesn't mean that the data has been written to the storage medium. But it does mean that the data is visible to other processes. So if another process goes to run after you've flushed your buffers, that data may not be on the storage medium but the other process should end up reading your data from the OS cache.

So you can still have a problem if, for example, the power fails after you get your sequence number. But delaying the unlocking doesn't help that case anyway.

I just noticed that merlyn made a good point about the danger of your read-cache being stale, but I don't think that applies in this particular case.

So my impression is that the explicit unlock isn't actually a problem in this specific case. I agree that it is a good thing to avoid as a general principle.

So my question is "Am I missing something?" I don't want to detract from the good points made. But I'm hoping for validation or specific refutation of my understanding here.

        - tye (but my friends call me "Tye")

Another small point. The end of the code should have a truncate in it:

...
print ON $ordernum;
truncate ON, tell;
close(ON);
[download]

AgentM Systems nor Nasca Enterprises nor Bone::Easy nor Macperl is responsible for the comments made by AgentM. Remember, you can build any logical system with NOR.

Yes, and while that handles the common (mistaken) case of
1. open for append
2. flock 2
3. append
4. unflock
5. close
that does nothing for
1. open for read/write
2. flock 2
3. read/write
4. unflock
5. repeat back to step 2 as needed

Because there's nothing in that sequence that refreshes the buffer between flock 2 and read, and yet someone else could have messed with the file to get your buffer and the file out of sync. So still, unflock is almost always a danger sign, even with the workaround.

In summary: unflock (flock 8) is not needed when you're about to close, and has to be handled very carefully if you're leaving the file open. Don't use unflock until you know why I say don't use unflock. {grin}

-- Randal L. Schwartz, Perl hacker

So, to sum it up, there is no 100% dependable synchronization method for a Perl program that needs to synchronize outside of the interpreter? Is this right?


AgentM Systems nor Nasca Enterprises nor Bone::Easy nor Macperl is responsible for the comments made by AgentM. Remember, you can build any logical system with NOR.