networker2149 has asked for the wisdom of the Perl Monks concerning the following question:

Is there a way to run parallel forks without using a loop or using Parallel::Fork::BossWorker? So I tried just using fork(), but that doesn't allow running forks in parallel. Next was Parallel::ForkManager, which wouldn't allow me to fork unless I was inside a loop of some sort. Maybe I'm missing something but I don't see the importance of having this as an arbitrary limitation. Last I tried Parallel::Fork::BossWorker which worked but doesn't seem to allow me to fork a subroutine and store the return values in a variable. I just want to fork the subroutines and store the return values in a hash. Here's a simplified example of my code. 

#!/usr/bin/perl
use strict;
use warnings;
no warnings "uninitialized";
use Parallel::Fork::BossWorker

my $data = "a";


my $fork_sub1 = Parallel::Fork::BossWorker->new(work_handler => \&sub1
+);
my $fork_sub2 = Parallel::Fork::BossWorker->new(work_handler => \&sub2
+);
my $fork_sub3 = Parallel::Fork::BossWorker->new(work_handler => \&sub3
+);
my $fork_sub4 = Parallel::Fork::BossWorker->new(work_handler => \&sub4
+);
my $fork_sub5 = Parallel::Fork::BossWorker->new(work_handler => \&sub5
+);
my $fork_sub6 = Parallel::Fork::BossWorker->new(work_handler => \&sub6
+);

$fork_sub1->add_work({data => "a"});
$fork_sub2->add_work({data => "a"});
$fork_sub3->add_work({data => "a"});
$fork_sub4->add_work({data => "a"});
$fork_sub5->add_work({data => "a"});
$fork_sub6->add_work({data => "a"});

$fork_sub1->process();
$fork_sub2->process();
$fork_sub3->process();
$fork_sub4->process();
$fork_sub5->process();
$fork_sub6->process();




sub sub1 {

my $variable = shift @_; 
print "sub 1\n";

return(0);
}

sub sub2 {

my $variable = shift @_; 
print "sub 2\n";

return(0);
}
sub sub3 {

my $variable = shift @_; 
print "sub 3\n";

return(0);
}
sub sub4 {

my $variable = shift @_; 
print "sub 4\n";

return(0);
}
sub sub5 {

my $variable = shift @_; 
print "sub 5\n";

return(0);
}
sub sub6 {

my $variable = shift @_; 
print "sub 6\n";

return(0);
}

[download]

Replies are listed 'Best First'.
Re: parallel forks
by zentara (Cardinal) on Sep 17, 2011 at 07:44 UTC
    You can just use a series of piped opens, and stuff their outputs into a hash. Here is a simple example. Instead of the while loop for reading output, you could make an IO::Select loop to read all the filehandles delivering your output. You can also specify different commands, by putting that into a hash too, instead of $cmd, you could have @cmds. 
    #!/usr/bin/perl
use warnings;
use strict;

my %pids;

my @to_be_processed = (1..20);

my $cmd = 'echo';

foreach my $p (@to_be_processed){ 
 
#   untested way of changing command for each fork
 #   $pids{$p}{'cmd'} = shift @cmds, 
 #  $pids{$p}{'pid'} = open($pids{$p}{'fh'}, "  $pids{$p}{'cmd'}    $p
+ 2>&1 |");

# simpler example 
 $pids{$p}{'pid'} = open($pids{$p}{'fh'}, "$cmd $p 2>&1 |");
}


# can alternatively use an IO::Select object here to collect
# data from filehandles
foreach my $key(keys %pids){ 
  my $fh = $pids{$key}{'fh'};
  while (<$fh>){
    print $_;
    #stuff data here into a return hash
   }
}

# prevent zombies
foreach my $key(keys %pids){
   waitpid($pids{$key}, 1);
}

print "done\n";

    [download]
    I'm not really a human, but I play one on earth.
    Old Perl Programmer Haiku ................... flash japh
[reply]
[d/l]
Re: parallel forks
by ikegami (Patriarch) on Sep 17, 2011 at 02:49 UTC

    If you want to do the same thing multiple times, you need some kind of loop. I know Parallel::ForkManager has the capability to send back data to the parent. See the "Data structure retrieval" example.

[reply]

      that's the problem. I think I want the forks to do different things, but it seems like none of the modules will do that in parallel and allow me to return data from the forked subroutines.

[reply]
        Why would you ever spawn more than one child if all the children were forced to do the same thing!? It's obviously not true that they can't do different things.
[reply]

Back to Seekers of Perl Wisdom