in reply to Atomic use of unlink() and SQL DELETE? How?

But how can I simulate an unlink without actually deleting the file?

Could you rename the file to be deleted, then perform the DB transaction and finally only delete the file once the DB transaction is completed? If the transaction fails or rolls back, you rename the file back.

And what to do if the storage filesystem (SAN) or database server goes haywire in the middle of the transaction and drops out?

If you get a major failure of this type, you would need some kind of integrity check that scans the DB looking to match the filenames and if one is not found, it looks for the renamed version and restores (renames) it to match the DB. This would be run as part of the DB startup/recovery procedure.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
  • Comment on Re: Atomic use of unlink() and SQL DELETE? How?

Replies are listed 'Best First'.
Re^2: Atomic use of unlink() and SQL DELETE? How?
by robins (Acolyte) on Feb 09, 2006 at 13:33 UTC

    Even though the rename method seems useful, you're still not sure the file is deletable, or what? As you noticed, I'm using -r to determine if I can delete the file. Is this the correct method? Or should I be checking the directory instead? Seem to recall reading something somewhere that the ability to delete a file is determined by checking write access on the containing directory. Am I wrong?

    How would this rename trick work if I try to delete multiple objects by calling the function multiple times? If I get a rollback in the middle it can't really rollback the entire unit of work because some of the files are already deleted...

      1. Even though the rename method seems useful, you're still not sure the file is deletable, or what?

        The answer to this, and several of the other questions will depend on who's SAN you are using, which platform etc. However, as a general statement, if an application needs to delete files, then it should run under an account with that privilege; or the files should be created so that the account under which it runs will be able to.

        What I'm saying here is that with the exception of general purpose system utilities, applications shouldn't need to check these privileges. It is really an administrative problem to ensure that the appropriate privileges are accorded to the relevant userids/groups and it should be enough to ensure that the application(s) responsible for creating and manipulating these files run under the correct accounts and set the file attributes correctly.

      2. As you noticed, I'm using -r to determine if I can delete the file. Is this the correct method?

        That will depend upon the SAN you are using and the host OS you are running under, and will be different depending upon whether you are using a *nix-like OS, Win32 or some other variant.

      3. Or should I be checking the directory instead?

        See above.

      4. Seem to recall reading something somewhere that the ability to delete a file is determined by checking write access on the containing directory. Am I wrong?

        That sounds vaguely correct for *nix FSs, but I know little about them. The picture is rather more complex on Win32. And it could be different again depending upon which SAN you are using, regardless of which OS you are running on.

        You will need to enquire after the SAN docs, and probably experiment to find out the answers.

      5. How would this rename trick work if I try to delete multiple objects by calling the function multiple times?

        If I get a rollback in the middle it can't really rollback the entire unit of work because some of the files are already deleted...

        You would need to structure your application differently. Indeed, if you are doing transaction-based work, you will need to anyway.

        For the rename mechanism to work, you would need to accumulate the names of the (renamed) files in the code calling the function until you do a commit and only delete the files once you have confirmation of the commit.

        Another potential problem is other applications/instances going looking for the files (by their original names) after you've renamed them but prior to the commit, but this should be taken care of by your DB taking a write lock on the records to be deleted and holding it until the transaction is committed or rolled back. It is just important to do the rename after the delete of the associated record.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.