Re^2: Parallel::ForkManager run_on_finish exit code handler

Okay, I spoke too soon. Accessing a hash defined in the parent IS read/writable from run_on_finish().

my %rc_hash         = ();
my $MAX_PROCESSES    = 120;
my $pm = new Parallel::ForkManager($MAX_PROCESSES);
    
$pm->run_on_finish(
     sub { 
      my ($pid, $exit_code, $ident) = @_;
         print "run_on_finish: $ident (pid: $pid) exited " . "with cod
+e: [$exit_code]\n";
    $rc_hash{$pid} = $exit_code;
         }
);

$pm->run_on_start(
      sub {
          my ($pid,$ident)=@_;
          print "** $ident started, pid: $pid\n";
       }
 );

for(my $i=0; $i< @hosts; $i++){
          $pm->start($hosts[$i]) and next;   
           system(@some_command);
           $return_code = $? >> 8;
           $pm->finish($return_code);
}

foreach my $key (keys %rc_hash){
      print "$key => $rc_hash{$key}\n";
}
[download]

I made an incorrect assumption.. What I was trying to do earlier is use a database handle from run_on_finish() and that didn't work, so I thought it applied to all other variables. I am not sure why accessing a hash would work, but accessing a db handle wouldn't. Is this possible?

my $dbh    = DBI->connect("DBI:mysql:database=mydb;host=$DB", "user", 
+"pass", {'RaiseError' => 1});
my $MAX_PROCESSES  = 120;
my $pm = new Parallel::ForkManager($MAX_PROCESSES);
    
$pm->run_on_finish(
     sub {
            my ($pid, $exit_code, $ident) = @_;       
            print "run_on_finish: $ident (pid: $pid) exited " . "with 
+code: [$exit_code]\n";
    insert_into_db(\$dbh, $pid, $exit_code);      
+    
          }
);
     $pm->run_on_start(
               sub {
                   my ($pid,$ident)=@_;
                   print "** $ident started, pid: $pid\n";
               }
               );

    for(my $i=0; $i< @hosts; $i++){
          $pm->start($hosts[$i]) and next;   
           system(@some_command);
           $return_code = $? >> 8;
           $pm->finish($return_code);
        }
$dbh->disconnect();

sub insert_into_db{
  my $dbhdl     = shift;
  my $pid       = shift;
  my $ret_code  = shift;

  $$dbhdl->do(INSERT INTO system_results ... .. );
}
[download]

If I can't use the db handle from run_on_finish(), I can still add the return code to %rc_hash and process them after $pm->finish(). Is this the right thing to do? thanks for the speedy responses. I hope I explained my problem a little better this time around.

Comment on Re^2: Parallel::ForkManager run_on_finish exit code handler Select or Download Code

Replies are listed 'Best First'.
Re^3: Parallel::ForkManager run_on_finish exit code handler by ikegami (Patriarch) on Sep 08, 2009 at 16:57 UTC
What I was trying to do earlier is use a database handle from run_on_finish() The child inherited the database handle and closed it. P::FM uses `exit` when it should probably use POSIX's `_exit`. `finish` is simply a call to `exec`, so if you call `_exec` instead of `finish`, that problem should go away. That means you should replace `for(my $i=0; $i< @hosts; $i++){ $pm->start($hosts[$i]) and next; system(@some_command); $return_code = $? >> 8; $pm->finish($return_code); }` [download] with `use POSIX qw( _exit ); for(my $i=0; $i< @hosts; $i++){ $pm->start($hosts[$i]) and next; system(@some_command); $return_code = $? >> 8; _exit($return_code); }` [download] Of course, `system` + `_exit` is a silly way of doing `exec`, so you get: `for my $host (@hosts) { $pm->start($host) and next; exec(@some_command); }` [download] Error checking added: `for my $host (@hosts) { $pm->start($host) and next; exec(@some_command); print(STDERR "exec failed: $!\n"); _exit($!); }` [download] That also fixes the bug where you'd return zero (success) when the child died from a signal. Update: Clean up.	[reply] [d/l] [select]
Re^4: Parallel::ForkManager run_on_finish exit code handler by sojourn548 (Acolyte) on Sep 08, 2009 at 19:25 UTC
ikegami - Thanks. The following code works. `for my $host (@hosts) { $pm->start($host) and next; exec(@some_command); print(STDERR "exec failed: $!\n"); _exit($!); }` [download] The database connection is now persistent in run_on_finish(), and I am able to insert the return code into the db from the callback routine. Thinking of the Chinese proverb.. give a man a fish.. teach a man to fish.. Rather than me just blindly accepting the code improvement, can you explain how run_on_finish() gets the return code without having $return_code passed in as either $pm->finish($return_code) or _exit($return_code) ? Also, why is 'exec' preferred over 'system + _exit' ? I thought exec executes a system command and never returns.. I am surprised that this works actually.	[reply] [d/l]
Re^5: Parallel::ForkManager run_on_finish exit code handler by ikegami (Patriarch) on Sep 09, 2009 at 00:25 UTC
Your premise that the child process doesn't call `exit` is wrong. Either it exited properly by calling `exit` with an exit code (setting `$?`'s high byte) or it was terminated by a signal (setting `$?`'s low byte). There's no other way for it to exit. I thought exec executes a system command and never returns.. Same goes for the call to `exit` it's replacing. Also, why is 'exec' preferred over 'system + _exit' ? You're creating a process whose sole purpose is to do nothing but create a process, and you're doing that for as many hosts as you have in your array. It's wasteful! It also requires more code and introduces problems (refer to the bug I mentioned it fixed).	[reply] [d/l] [select]