in reply to Re: Frustration with changing lock type
in thread Frustration with changing lock type

Thanks for the reply Hugo. I went with something like what you suggest. Added a test before we try to aquire the LOCK_EX to make sure the lockfile doesnt exist or is at least 3 seconds old, and double checking that the process data stays the same after the LOCK_UN and LOCK_SH. The append logic I didnt quite follow so I didnt go that way. Here is what I have: 

sub get_lock {
    my ($lock_dir,$name)=@_;
    my $debug=1;

    $name=~s/[^-\w.#!\@~=+%\$]//g;
    my $lockfile=catfile($lock_dir,$name.'.lock');

    print "Trying lockfile $lockfile\n";
    if (-e $lockfile and (time()- -M $lockfile)<3) {
        print "\tLockfile exists and is very recent skipping for now\n
+";
        return;
    }
    sysopen(my $FH, $lockfile, O_RDWR | O_CREAT)
        or do { warn "can't open $lockfile: $!" if $debug;
                return;
              };
    # autoflush $FH
    select( (select($FH), $|++)[0] );
    my ( $time, $process, $lname );

    if (flock( $FH, LOCK_EX | LOCK_NB )) {
        ( $time, $process, $lname )=split /\|/,join "",<$FH>;
        seek $FH, 0, 0 or die "Failed rewind:$!";

        if ($debug) {
            if ($process) {
                print "\tLockfile appears to be abandonded by Process 
+#$process started at $time\n"
            } else {
                print "\tLockfile appears to be unprocessed\n"
            }
        }
        my $lock_msg=join("|", iso_time(), $$, $name)."\n";

        print $FH $lock_msg;
        truncate($FH, tell($FH)) or die "Failed to truncate:$!";
        flock($FH, LOCK_UN) or die "sharedlock: $!";
        flock($FH, LOCK_SH|LOCK_NB) or do {
            print "\tFailed to relock!\n" if $debug;
            return;
        };
        seek $FH, 0, 0 or die "Failed rewind2:$!";
        my $msg=<$FH>;
        unless ($msg eq $lock_msg) {
            print "\tWhoops, (harmless) race condition on $lockfile!\n
+";
            return
        }
        print "File Locked! : $lock_msg";
        return OnDestroy {
                            print "*** Finished with and removing $loc
+kfile ***\n";
                            close $FH or die "Failed to close \$FH:$!"
+;
                            unlink $lockfile or die "Failed to unlink 
+$lockfile\n";
                            undef $FH;
                         };
    } elsif (flock($FH, LOCK_SH)) {
        ( $time, $process, $lname )=split /\|/,join "",<$FH>;
        print "\tLockfile appears to be locked by Process #$process at
+ $time\n"
            if $debug;
    } else {
        print "\tFailed to get lock! Not sure why.\n";
    }
    return
}

[download]

Yes its a bit wordy, id be interested to see a less clunky implementation if anyone feels inclined.

Incidentally there is no need to wait to get the exclusive lock. Under normal situations this will be polling a table in a DB every 60 seconds or so, and then processing any records it finds there. So if the lock file is too young returning to try the next record is better.

Cheers


---
demerphq

    First they ignore you, then they laugh at you, then they fight you, then you win.
    -- Gandhi

Replies are listed 'Best First'.
Re: Re: Re: Frustration with changing lock type
by hv (Prior) on Apr 14, 2004 at 12:47 UTC

    Hmm, I think you still have a race in the OnDestroy: if I understand flock() correctly, the lock is released as soon as you close the filehandle, so the unlink() happens after the lock has been released.

    If I remember right, Win32 doesn't let you unlink a file while anyone still has a handle open on it, so maybe it would be better instead to truncate the file to zero bytes on completion, which is something you can do with an open filehandle.

    Hugo

[reply]
      If I remember right, Win32 doesn't let you unlink a file while anyone still has a handle open on it

      No, that is just a type of sharing that you can specify. The C RTL defaults to opening files sharing both read and write access but not sharing rename/delete access.

      If you want to delete a file while you have it open in Perl, you can use Win32API::File to open the file and specify that you want to allow FILE_SHARE_DELETE.

      - tye        

[reply]

      Hmm, I think you still have a race in the OnDestroy: if I understand flock() correctly, the lock is released as soon as you close the filehandle, so the unlink() happens after the lock has been released.

      Hmm, I see what you mean. The thing is that the OnDestroy doesnt happen until after the record that this job is about is deleted from the DB. So once the deletion occurs the lockfile is more or less irrelevent as no other job can now get it. We requery the queue after every successful task, so I _think_ the race condition is covered. But maybe im wrong, and maybe there is even a better way to go about this.

      Actually the more i think about it the more I think I am wrong. Hmm.


      ---
      demerphq

        First they ignore you, then they laugh at you, then they fight you, then you win.
        -- Gandhi
[reply]
[d/l]

        The issue is not about _this_ record, but that you may have screwed up another process trying to handle the _next_ one. I think this would happen only if you took more than 3 seconds to process the record, so that the next process may see that the timestamp is old and grab it anyway: if it gets the LOCK_EX in the interval between you closing the file and unlinking it, it will merrily truncate and rewrite the file without knowing that anything has changed under its feet.

        It may be that the effect under Win32 would be that the unlink() fails, so that the new process really does have the actual file still. On a *nix system, I think it would instead have a locked filehandle on a file that is no longer linked on the filesystem, and would therefore be in danger of clashing with the next process to come and get a lock.

        However truncating instead should (I think) be safe either way round.

        Hugo

[reply]

In Section Seekers of Perl Wisdom