Using Parallel::ForkManager I had a piece of code with a run_on_finish() method that was never executed when I tried to run with the maximum number of proces set higher than 0. If the max_procs was 0 (disabling the fork() and using some emulation) than the run_on_finish callback ran just fine.

After much head scratching and source reading it looks like P:FM doesn't install any signal handling to actually catch SIG{CHLD} and thus run the run_on_finish() callback in the parent. It does try to call waitpid() sometimes but really only when it starts a new process (start() method).

I believe that P:FM is losing its children to the default SIG{CHLD} handler. However, the docs don't state that anything special is neccesary to do use run_on_finish().

The code below fixed my issue. 

my $pm = Parallel::ForkManager->new(2);

$SIG{CHLD} = sub{  Parallel::ForkManager::wait_children($pm) };

[download]

Shouldn't P:FM install a signal handler for SIGCHLD? Also, how does the out of the box example (callback.pl in source) work at all without a signal handler?

I am not (to my knowledge) installing signal handlers or otherwise calling fork besides the odd backtick.

Thoughts?

EDIT: Problem re-production. The code works with max_procs set to 0 but not any higher than 0 due to run_on_finish() never being called

In this code I'm not doing any crazy process management. I do manage my workload but I expect run_on_finish() when a process finished to be called per the docs. The key point here is run_on_finish() generates new work. If I call 'ps' I see that the children are getting cleaned up by the system but run_on_finish() was never called.

#!/usr/bin/env perl -w

use strict;
use Parallel::ForkManager;

use 5.012;

my $max_procs = 5;

my @name_queue   = qw( bot403 );
my @future_names = qw( Fred Jim Lily Steve Jessica Bob Dave Christine 
+Rico Sara );


my $pm =  new Parallel::ForkManager($max_procs);

# Track children in progress
my %working;

# Setup a callback for when a child finishes up so we can
# get it's exit code
$pm->run_on_finish(
  sub { my ($pid, $exit_code, $ident) = @_;
    print "** $ident just got out of the pool ".
      "with PID $pid and exit code: $exit_code\n";
    my $new = shift @future_names;
    # Done with this work unit.
    shift @name_queue;
    # Get future work to do (oversimplification of my real problem)
    push @name_queue, $new if $new;
    # Mark this entry as no longer in progress
    delete $working{$ident};
  }

);

$pm->run_on_start(
  sub { my ($pid,$ident)=@_;
    print "** $ident started, pid: $pid\n";
  }
);

$pm->run_on_wait(
  sub {
    print "** Have to wait for one children ...\n"
  },
  10
);

while( @name_queue || scalar keys %working ){
  
  # Screen out entries in progress so we don't duplicate work.
  my @to_process = grep { ! $working{$_} } @name_queue;

  if( !@to_process ){
   say "Waiting for children to finish to find new names";
   sleep 10;
   next;
  }

  foreach my $child (@to_process){
    $working{$child} = 1;
    $pm->start($child) and next;

    # This code is the child process
    say "This is $child";
    sleep 3+rand(5);
    $pm->finish(0); # pass an exit code to finish
  }
}

print "Waiting for Children...\n";
$pm->wait_all_children;
print "Everybody is out of the pool!\n";

[download]

In reply to Parallel::ForkManager run_on_finish() bug? by bot403

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.