geektron has asked for the wisdom of the Perl Monks concerning the following question:

i got notification from an old client today that the search script suite i wrote ( a data file builder run via cron and a seach data file script ) stopped building the new datafile. i found an obvious error ( never let a non-perl user add an email address - the foo\@bar part ALWAYS gets forgotten! ) and thought i could call it fixed.

but no. it's not fixed. after about 2 hours of extra debugging, including wrapping everything in  eval loops to catch errors, the bloody thing still died. abruptly. with no error messages.

so i pulled the site to my local machine, and ran the databot, and everything worked like a charm.

which got me thinking -- the ISP's looking for 'runaway processes' and killing them.

so i'm wondering if it's feasible to try and catch  $SIG{KILL} in the script and try to ignore it.

i tried this, to no avail:

local $SIG{KILL} = \&killed; ### some code sub killed { print "I got killed\n"; }
an END block doesn't get executed when the script is killed. i tried:
END { print "I got killed! \n "; }
but that didn't work either

i have other ideas for getting around this apparent rate cap . . . but i would like to know that the script is dying because of REAL errors, and not getting killed by some other process.

Replies are listed 'Best First'.
Re: getting around an ISPs processing cap
by knobunc (Pilgrim) on May 09, 2001 at 16:20 UTC

    The suggestions above about using a checkpoint file are good. But as a cheap hack can you just fork periodically and kill the parent after the fork leaving the child running the same thread of execution? I am not sure if they count parent time against a child process. Also this may not work depending upon what you need to keep open across the fork.

    my $fork_period = 30*60; # 30 minutes my $forktime = time + $forkperiod; while (1) { # Do my happy processing. # This must loop periodically (at most once a minute) # otherwise you may badly miss your forktime and get # killed. # We don't want to fork too often (or they might think # we are a runway and kill us). if (time > $forktime) { # Kill the parent exit if fork(); $forktime = time + $forkperiod; } }

    BTW they might be using resource limits (man getrlimit or ulimit) to kill the process. That may be smart enough to go across children.

    -ben

      Actually, that's a pretty good technique (which I was thinking of before I scrolled down to see yours!).

      The limit stuff applies per process, so the kids would get a reset counter for all of them.

      About the only thing this messes up is the "who's yer daddy?" situtation. In fact, they'd be children of init(8), so they'd charge not to your process, but to the great granddaddy when they died. Pretty slick.

      If the trigger was strictly on CPU time, you could just do this:

      use List::Util qw(sum); while (1) { if (sum(times) > 30) { # have I used more the 30 CPU seconds? fork and exit; # carry on, my wayward son, there'll be peace when +you are done... } ... rest of processing here ... }

      -- Randal L. Schwartz, Perl hacker

      BTW they might be using resource limits (man getrlimit or ulimit) to kill the process. That may be smart enough to go across children.

      hmm...  man getrlimit says:

      Limits on the consumption of system resources by the current process +and each process it creates may be obtained with the getrlimit() call +, and set with the setrlimit() call.
      sounds like child processes will get nuked as well. ( i know the ISP's using FreeBSD, and so am i. man page straight outta FreeBSD 4.2 ).

      this is exactly the kind of stuff i'm worried about

      i can't get in an change the frequency of the crontab, or anything remotely useful. ( ISP requires a cron.sh file - which contains jobnames -- , and THEY control the crontab )

      so forking won't really be an option . . .

      i need to do more research on how the ISP is doing the resource limiting, i guess. I was hoping for a quick way out of the problem.

      BTW -- i'm not talking long limits. the script gets killed in a matter of minutes. single digit minutes.

Re: getting around an ISPs processing cap
by busunsl (Vicar) on May 09, 2001 at 14:16 UTC
    The problem is that you can't catch the KILL signal.
    It is caught by the OS and will not even reach you program.
    But if you are lucky the signal will be TERM, which you can catch.
Re: getting around an ISPs processing cap
by clintp (Curate) on May 09, 2001 at 15:49 UTC
    If you're getting KILL, then forget about it. It can't be ignored or caught. How is it determining how old your process is? Would an occasional fork() (letting the parent die) help?

    A better technique anyway might be to write an occasional checkpoint file out to disk and to run the job more frequently in cron. If the task is already running do nothing. But if the task is gone, and the checkpoint file is there, pick it up and continue processing.

Re: getting around an ISPs processing cap
by zigster (Hermit) on May 09, 2001 at 15:50 UTC
    You could write it as two processes parent and child connected via a pipe. Do all the hardwork in the child storing results and status in the parents. When the child dies restart it from the parent from where it left off from the status info.
    --

    Zigster