Takamoto has asked for the wisdom of the Perl Monks concerning the following question:
This is my first attempt to process things in parallel with Parallel::ForkManager on a server. I have several subrutines to collect data through APIs. I want to perform it in parallel and then merge the results. This is my script, not elegant of course, but it runs. As I do not see a huge difference in performance (time) in running things in parallel with this script or running the single subrutines one after the other (the script let me save ~1/3 of the time), just wanted to ask for your wisdom about my script
use Parallel::ForkManager; my $max_procs = 6; my @names = qw( 0 2 3 4 5 0 ); my @DataStructure; my $pm = Parallel::ForkManager->new($max_procs, @ARGV); $pm->run_on_finish( sub { my ($pid, $exit_code, $ident, $exit_signal, $core_dump, $data_struct +ure_reference) = @_; my @results= @$data_structure_reference; if (@results){ push (@DataStructure, @results); } }); foreach my $child ( 0 .. $#names ) { my $pid = $pm->start($names[$child]) and next; my @results; if ($child eq 1){ @results=getResultsAPI_1(); } elsif ($child eq 2){ @results=getResultsAPI_2(); } elsif ($child eq 3){ @results=getResultsAPI_3(); } elsif ($child eq 4){ @results=getResultsAPI_4(); } elsif ($child eq 5){ @results=getResultsAPI_5(); } elsif ($child eq 6){ @results=getResultsAPI_6(); } $pm->finish($child, \@results); } $pm->wait_all_children;
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re: Parallel::ForkManager right approach
by roboticus (Chancellor) on Oct 13, 2019 at 17:59 UTC | |
|
Re: Parallel::ForkManager right approach
by davido (Cardinal) on Oct 14, 2019 at 05:09 UTC | |
|
Re: Parallel::ForkManager right approach
by karlgoethebier (Abbot) on Oct 14, 2019 at 15:49 UTC |