rhesa has asked for the wisdom of the Perl Monks concerning the following question:

Up until today, I was a happy user of Proc::PID::File. But this morning, one of my servers ran out of disk space on /var, and that module wasn't able to write out its pidfile anymore. The result was a process table full of long-running cron jobs, bringing the machine down due to excessive swapping.

Is there a better alternative for Proc::PID::File that will gracefully handle reboots *and* running out of disk space?

I want to be certain only one job runs at a time, even if I'm not managing my disk space properly :)

Update: obviously, a code snippet is in order (sorry, grep): 

#!/usr/bin/perl

use Proc::PID::File;
die "Already running!" if Proc::PID::File->running;

# rest of script follows

[download]

Another update: Just so I don't come off like a complete idiot, here's what happened to this machine. It has a replicating MySQL slave running, and somewhere during the night, there was an error with it. The result was that the slave kept recording the relay-bin logs, but didn't process them. Since our data changes a lot, the slave quickly filled up the disk. I don't think monitoring tools would have avoided me waking up to a thrashing box.

Replies are listed 'Best First'.
Re: Whoa! looking for a better pidfile module
by grep (Monsignor) on Dec 14, 2006 at 01:55 UTC

    No offense but this doesn't really sound like a problem with the module. The truth is you should be managing your disk space properly. Look into Nagios it will monitor disk space (among many other things) for you.

    Looking at current source, I see a die when file can not be written, so I assume you are evaling the Proc::PID::File->running() call and disregarding the result. But I can't tell for certain because you didn't post any code. It does die for me when I try to assign it to my CD.

    IIRC, you can also clear your program's pid files at the beginning of reboot in rc.sysinit

    UPDATE:
    Since our data changes a lot, the slave quickly filled up the disk. I don't think monitoring tools would have avoided me waking up to a thrashing box.

    Sure it could - I've written many Nagios Plugins. All of them before helpful modules like Nagios::Plugin. Writing Nagios plugin are not difficult, that's why I like it. You could easily monitor the number of file to be processed. Not to mention that Nagios also has CPU and Memory (and IIRC swapping monitors) already built in.

    grep
    1)Gain XP 2)??? 3)Profit

[reply]
[d/l]
[select]
      No offense taken, I know I'm a bit lazy. I updated my original node with a code snippet: I do die when running() returns false. I think what happened was that it can create the file, but not write the pid into it.

      Reading the source for my installed version (1.24) now, I see that it doesn't check the return value of syswrite, and I'm pretty sure that's the bug that bit me.

      So, I'm off to post a bug report and a patch :-)

      BTW, I have been meaning to look into Nagios. I have some 16 machines under my care now, so it's not all that strange I missed this. A man without tools isn't much more than a monkey ;^)

[reply]
[d/l]
[select]
        "I have been meaning to look into Nagios. I have some 16 machines under my care now.."

        I'd just like to throw in a plug for Big Brother. I've been using it for several years now, and currently we're running about 15 different tests on 600+ hosts - and it never misses a beat.

        It's extremely easy to setup and configure, and also quite easy to customise to specific needs. I can't say if it's better or worse than Nagios, as I've not used that. Just offering an alternative :)

        Cheers,
        Darren :)

[reply]
Re: Whoa! looking for a better pidfile module
by merlyn (Sage) on Dec 14, 2006 at 04:51 UTC
[reply]

Back to Seekers of Perl Wisdom