Program unexpectedly terminates

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I have a simple crawler that is based on LWP::UserAgent. It loops through a bunch of database records and fetches and processes web content related to each record. The problem is that the program will unexpectedly terminate at some point with not indication of the reason. This could happen a few minutes or a few hours into the processing. And when this happens the exit code of the program is 0. I am using Carp::Alway to enable stack-tracing and have added an END block as well as a DESTROY method to my crawler and neither of those produces any output when it terminates. I have also tried to override CORE::Global::exit. My code never calls exit and I'm positive I'm not accidentally short-circuiting my loop, otherwise the die after run would get called. Below is a skeleton of my code. Is there something else I can use to try and narrow this down?

#!/usr/bin/env perl
use 5.016;
use warnings;
use Carp::Always;

BEGIN { *CORE::GLOBAL::exit = sub { die } }
END { say "END block"; }

my $crawler = Crawler->new;
$crawler->run;

die;

package Crawler {
    use parent 'LWP::UserAgent';
    # use a bunch of other modules for the processing.

    sub new {
        my $self = $class->SUPER::new(%args);
        $self->{ids} = (1..100);  # These normally are pulled from a D
+B.
        return $self;
    }

    sub run {
        my $self = shift;
        my @idx = shuffle 0 .. -1 + @{ $self->{ids} };
        my $cur = 0;
        for my $i (@idx) {
            my $id = $self->{ids}[$i];
            printf "%d/%d: %s (%d)\n", ++$cur, 0+@idx, $id, $i;
            # Fetch and process stuff related to this id.
        }
    }

    sub DESTROY { say "Crawler destroyed" }
}
[download]

Comment on Program unexpectedly terminates Download Code

Replies are listed 'Best First'.
Re: Program unexpectedly terminates by tobyink (Canon) on Nov 19, 2012 at 09:29 UTC
It would be helpful if the sample code you provided actually compiled (undefined `$class`, `%args` and `&shuffle` prevents compilation with strict subs/vars). Fixing those issues... Read more... (1112 Bytes) ... the code seems to execute correctly here. `perl -E'sub Monkey::do{say$_,for@_,do{($monkey=[caller(0)]->[3])=~s{::}{ }and$monkey}}"Monkey say"->Monkey::do'`	[reply] [d/l] [select]
A reply falls below the community's threshold of quality. You may see it by logging in.
Re: Program unexpectedly terminates by ColonelPanic (Friar) on Nov 19, 2012 at 08:31 UTC
The code you have shown has a problem. You are assigning an array (1..100) as the value in a hash. You need an array reference for that (use brackets instead of parentheses). However, I doubt that is the real problem with your code--it was probably just a mistake when you created the simplified example. Also, I tried your CORE::GLOBAL::exit override, and it didn't do anything--it only got worked if I explicitly called the exit() command, which your code does not. I think your problem must be in the processing that you are not showing. It could be failing on the same one each time. Because you are processing in a random order, it would take varying amounts of time for that condition to be reached. I suggest some good, old-fashioned debug print statements within your processing loop. Log the id of each record you are looking at, and record the success of each significant operation. That should help you to know where it fails. When's the last time you used duct tape on a duct? --Larry Wall	[reply]
Re^2: Program unexpectedly terminates by Anonymous Monk on Nov 19, 2012 at 08:48 UTC
The code you have shown has a problem. You are assigning an array (1..100) as the value in a hash. You need an array reference Yeah, yeah. That's obviously not working code and I was trimming down my real program to just show the basic structure. Also, I tried your CORE::GLOBAL::exit override, and it didn't do anything--it only got worked if I explicitly called the exit() command, which your code does not. I said in my OP I never directly call exit. But my unstated assumption is that one of the many CPAN modules I am using might be calling it. I think your problem must be in the processing that you are not showing. It could be failing on the same one each time. Because you are processing in a random order, it would take varying amounts of time for that condition to be reached. I'm intentionally not wrapping anything in eval, so if something in my code does fail, it should bomb out loudly. It never does. I suggest some good, old-fashioned debug print statements within your processing loop. Log the id of each record you are looking at, and record the success of each significant operation. That should help you to know where it fails. The print statement I have in the example already prints before the processing starts, so I see which record it stops at. It's never the same one. Plus, as I mentioned above, any error in my code would die loudly. But I'm not sure why the program can terminate with exit value of 0, yet the END block is never called. So my assumption is that one of the CPAN modules is at fault, but whatever is the cause, I was hoping there was some module that would perform the necessary overrides and tell me what's calling exit.	[reply]
Re^3: Program unexpectedly terminates by ColonelPanic (Friar) on Nov 19, 2012 at 09:09 UTC
Thanks for the additional information. I didn't understand that you were trying to trap explicit calls to exit(). That makes sense, then. Playing around more with the BEGIN block, I found that an exit() statement within another package was only overridden if the BEGIN block came before the package definition. Do you have this block before you include all of your modules? Also, would Test::Trap be helpful? When's the last time you used duct tape on a duct? --Larry Wall	[reply]
Re^4: Program unexpectedly terminates by Anonymous Monk on Nov 19, 2012 at 09:20 UTC
Re^5: Program unexpectedly terminates by ColonelPanic (Friar) on Nov 19, 2012 at 09:56 UTC
Some notes below your chosen depth have not been shown here
Re: Program unexpectedly terminates by aitap (Curate) on Nov 19, 2012 at 14:37 UTC
Assuming you have an UNIX-like system, try running your program under strace. This will produce a lot of text, but probably last lines will show you the reason. Sorry if my advice was wrong.	[reply]
Re: Program unexpectedly terminates by Anonymous Monk on Nov 19, 2012 at 12:40 UTC
Anonymously speaking, the place to start is by adding `print STDERR I am here now` messages throughout the code. Watch for the last one.	[reply]
Re: Program unexpectedly terminates by Anonymous Monk on Nov 19, 2012 at 19:03 UTC
As a shot in the dark, I added $SIG{PIPE} = 'IGNORE'; and haven't seen the problem reoccur in 7 hours, so it's looking like that solved the problem. Now I have to figure out if the culprit is LWP::UserAgent (and it's many dependencies) or Net::Proxy::Type, or IO::Socket::Socks. Then produce a repeatable test case and properly report it as a bug.	[reply]