Re^3: running a function for different list parallely using fork

One thing that is not clear to me, how it will handle the running of regressions in case number of lists in @teslLists is odd (i.e 3 or 5 or 7). because at last, only one list will be left. so will below line will run only last list even though we have passed 2 to run in parallel.

my $pm = Parallel::ForkManager->new( 2 );
[download]

Comment on Re^3: running a function for different list parallely using fork Download Code

Replies are listed 'Best First'.
Re^4: running a function for different list parallely using fork by ikegami (Patriarch) on Mar 22, 2023 at 19:50 UTC
P::FM forks a child for each "task". If you provide a limit (e.g. the `2`), it waits for children to finish before forking if forking would bring the number of children over the limit. `use v5.14; use warnings; use Parallel::ForkManager qw( ); use Time::HiRes qw( sleep ); my $pm = Parallel::ForkManager->new ( 2 ); for ( 1 .. 9 ) { $pm->start and next; say "[$_] start"; sleep( $_ == 1 ? 4 : 0.3 ); say "[$_] done"; $pm->finish; } $pm->wait_all_children();` [download] Annotated output: `1 2 3 4 5 6 7 8 9 ----- --- --- --- --- --- --- --- --- --- [1] start 1 XXX [2] start 1 & 2 XXX XXX [2] done 1 XXX [3] start 1 & 3 XXX XXX [3] done 1 XXX [4] start 1 & 4 XXX XXX [4] done 1 XXX [5] start 1 & 5 XXX XXX [5] done 1 XXX [1] done [6] start 6 XXX [7] start 6 & 7 XXX XXX [6] done 7 XXX [7] done [8] start 8 XXX [9] start 8 & 9 XXX XXX [8] done 9 XXX [9] done` [download]	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re^4: running a function for different list parallely using fork
by ikegami (Patriarch) on Mar 22, 2023 at 19:50 UTC

P::FM forks a child for each "task". If you provide a limit (e.g. the 2), it waits for children to finish before forking if forking would bring the number of children over the limit.

use v5.14;
use warnings;

use Parallel::ForkManager qw( );
use Time::HiRes           qw( sleep );

my $pm = Parallel::ForkManager->new ( 2 );

for ( 1 .. 9 ) {
   $pm->start and next;
   say "[$_] start";
   sleep( $_ == 1 ? 4 : 0.3 );
   say "[$_] done";
   $pm->finish;
}

$pm->wait_all_children();
[download]

Annotated output:

                   1   2   3   4   5   6   7   8   9
           -----  --- --- --- --- --- --- --- --- ---
[1] start  1      XXX
[2] start  1 & 2  XXX XXX
[2] done   1      XXX 
[3] start  1 & 3  XXX     XXX
[3] done   1      XXX 
[4] start  1 & 4  XXX         XXX
[4] done   1      XXX 
[5] start  1 & 5  XXX             XXX
[5] done   1      XXX 
[1] done
[6] start  6                          XXX
[7] start  6 & 7                      XXX XXX
[6] done   7                              XXX
[7] done
[8] start  8                                  XXX
[9] start  8 & 9                              XXX XXX
[8] done   9                                      XXX
[9] done
[download]

[reply]
[d/l]
[select]