Using Parallel::ForkManager I had a piece of code with a run_on_finish() method that was never executed when I tried to run with the maximum number of proces set higher than 0. If the max_procs was 0 (disabling the fork() and using some emulation) than the run_on_finish callback ran just fine.

After much head scratching and source reading it looks like P:FM doesn't install any signal handling to actually catch SIG{CHLD} and thus run the run_on_finish() callback in the parent. It does try to call waitpid() sometimes but really only when it starts a new process (start() method).

I believe that P:FM is losing its children to the default SIG{CHLD} handler. However, the docs don't state that anything special is neccesary to do use run_on_finish().

The code below fixed my issue.

my $pm = Parallel::ForkManager->new(2);

$SIG{CHLD} = sub{  Parallel::ForkManager::wait_children($pm) };
[download]

Shouldn't P:FM install a signal handler for SIGCHLD? Also, how does the out of the box example (callback.pl in source) work at all without a signal handler?

I am not (to my knowledge) installing signal handlers or otherwise calling fork besides the odd backtick.

Thoughts?

EDIT: Problem re-production. The code works with max_procs set to 0 but not any higher than 0 due to run_on_finish() never being called

In this code I'm not doing any crazy process management. I do manage my workload but I expect run_on_finish() when a process finished to be called per the docs. The key point here is run_on_finish() generates new work. If I call 'ps' I see that the children are getting cleaned up by the system but run_on_finish() was never called.

#!/usr/bin/env perl -w

use strict;
use Parallel::ForkManager;

use 5.012;

my $max_procs = 5;

my @name_queue   = qw( bot403 );
my @future_names = qw( Fred Jim Lily Steve Jessica Bob Dave Christine 
+Rico Sara );


my $pm =  new Parallel::ForkManager($max_procs);

# Track children in progress
my %working;

# Setup a callback for when a child finishes up so we can
# get it's exit code
$pm->run_on_finish(
  sub { my ($pid, $exit_code, $ident) = @_;
    print "** $ident just got out of the pool ".
      "with PID $pid and exit code: $exit_code\n";
    my $new = shift @future_names;
    # Done with this work unit.
    shift @name_queue;
    # Get future work to do (oversimplification of my real problem)
    push @name_queue, $new if $new;
    # Mark this entry as no longer in progress
    delete $working{$ident};
  }

);

$pm->run_on_start(
  sub { my ($pid,$ident)=@_;
    print "** $ident started, pid: $pid\n";
  }
);

$pm->run_on_wait(
  sub {
    print "** Have to wait for one children ...\n"
  },
  10
);

while( @name_queue || scalar keys %working ){
  
  # Screen out entries in progress so we don't duplicate work.
  my @to_process = grep { ! $working{$_} } @name_queue;

  if( !@to_process ){
   say "Waiting for children to finish to find new names";
   sleep 10;
   next;
  }

  foreach my $child (@to_process){
    $working{$child} = 1;
    $pm->start($child) and next;

    # This code is the child process
    say "This is $child";
    sleep 3+rand(5);
    $pm->finish(0); # pass an exit code to finish
  }
}

print "Waiting for Children...\n";
$pm->wait_all_children;
print "Everybody is out of the pool!\n";
[download]