in reply to Re: Should I call waitpid when using Parallel::ForkManager to fork in an infinite loop?
in thread Should I call waitpid when using Parallel::ForkManager to fork in an infinite loop?

run_on_finish is called in the parent process immediately after reaping the child (to get its exit code). It surprises me that it would be called "much later" since the parent is constantly checking if a child has ended if you use the formula in the documentation.

The only reason it wouldn't be responsive is if you have slow code that executes in the parent. This would delay start getting called, which would delay checking if children have ended. Note that start will actually reap multiple children if more than one have ended.

  • Comment on Re^2: Should I call waitpid when using Parallel::ForkManager to fork in an infinite loop?
  • Select or Download Code

Replies are listed 'Best First'.
Re^3: Should I call waitpid when using Parallel::ForkManager to fork in an infinite loop?
by unlinker (Monk) on Aug 16, 2009 at 18:03 UTC
    Allow me to post the complete code to explain what I see happening. The context is a worker application that watches a beanstalk message queue, forks a child to handle the request and immediately goes back to watching the queue. Let me paste the code and then describe what I see: 
    #!/usr/bin/perl

use strict;
use warnings;

use Beanstalk::Client;
use Parallel::ForkManager;

my $clnt = Beanstalk::Client->new({
    server => '127.0.0.1:11300',
    debug => 1,
}) || die "Cannot Connect to Queue Manager";

my $pmgr = Parallel::ForkManager->new(20);

$pmgr->run_on_finish(
    sub {
        my ($pid, $exit_code, $ident) = @_;

        if ($clnt->delete($exit_code)) {
            print "Deleted Job $exit_code\n";
        }
        else {
            print "Error Deleting Job $exit_code: " . $clnt->error . "
+\n";
        }
    }
);

$pmgr->run_on_start(
    sub {
        my ($pid, $ident) = @_;

        if ($clnt->bury($ident)) {
            print "Buried Job $ident\n";
        }
        else {
            print "Error burying Job $ident: " . $clnt->error . "\n";
        }
    }
);

INFINITE_LOOP:
while (1) {

    sleep 5;

    my $job = $clnt->reserve(); #blocks until a message is found in qu
+eue
    if (! $job) {
        print "Error Reserving - Possible Deadline Approaching: " . $c
+lnt->error ."\n";
        next INFINITE_LOOP;
    }
    print "Reserved Job " . $job->id . "\n";
    
    $pmgr->start($job->id) and next;
    sleep 120; # Job work would go here
    $pmgr->finish($job->id);
}

exit;

    [download]
    When I run this code, here is what I find: (1) When the messages are coming at intervals of say 10 seconds, run_on_finish is never called. (2) After the 10th or 12th message or so, suddenly (no pattern that I can discern) run_on_finish is called repeatedly to reap all 10 (or 12) processes that have ended. (3) Again for about 10/12 messages, no run_on_finish and then suddenly again after 10/12 messages all finished tasks are reaped. Would appreciate if you could help me understand whats going on
[reply]
[d/l]

      Would appreciate if you could help me understand whats going on

      I've already explained:

      The only reason it wouldn't be responsive is if you have slow code that executes in the parent.

      In addition to intentionally slowing down the parent (sleep 5;), your parent blocks (my $job = $clnt->reserve();). This prevents it from reaping in a timely manner.

[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom