in reply to Avoiding race condition with sysopen

I'm after permanent files with unique and sequential file names.
Then enforce that by storing a sequence number somewhere, permanently. Make all programs that create files in that directory, use it.

Think of an autoincrement field in a database (why not use that? SQLite might be a decent and fast solution), or a setup with a little file holding the number, pretty much like the "web access counters" of a decade ago.

  • Comment on Re: Avoiding race condition with sysopen

Replies are listed 'Best First'.
Re: Re: Avoiding race condition with sysopen
by revdiablo (Prior) on Mar 05, 2004 at 19:57 UTC
    stor[e] a sequence number somewhere, permanently.

    That's exactly what I'm doing. My database is the filesystem. My "sequence file" is a directory full of plain files. Building all sorts of infrastructure to support what could be accomplished in 5 or 10 lines of code that just looks at the filesystem... seems kind of, well, overkill.

Re^2: Avoiding race condition with sysopen
by graff (Chancellor) on Jun 28, 2004 at 04:23 UTC
    Think of an autoincrement field in a database (why not use that? ...), or a setup with a little file holding the number, pretty much like the "web access counters" of a decade ago.

    To augment revdiablo's reply, if someone is already using a database for things related to the task at hand, then for sure, it makes better sense to use a database to create this sort of "guaranteed unique" identifier. But if there is no database facility on hand -- and it's just a matter of using the file system -- then installing a database just for this purpose is more trouble than it's worth.

    On the other hand, using a "little file" somewhere doesn't help much either, really -- it just adds another few ways that the "guaranteed unique" naming could fail (I've seen it happen).

    For instance, if there is any sort of error when writing the lastest (or next) increment to that little file, it might end up unchanged, or worse yet, empty -- and unless you're really careful about that, you could start you're sequence over at 0 -- perhaps multiple times -- without noticing until it's too late.

    When you are careful about it, you'll find yourself looking at the directory contents anyway, to determine the latest-used file name (as in the OP) or at least to confirm that a file name you make up on the fly does not already exist (as in the solution proposed by tachyon), so you might as well just stick with that -- it's both necessary and sufficient.

    If I were looking for sequential file names, as suggested by the OP, I would start with

    $last = (sort { $b cmp $a } readdir( DIR ))[0];
    so that I easily get the latest item in the directory and increment from there.

    But in any case, if there were a chance that the script would be run in concurrent processes on the same directory, I'd add a semaphore file, so that the steps of checking directory contents and creating the new file would be an atomic unit, doable by only one process at a time.