in reply to Re^2: Parallel::ForkManager How to wait when 1st 10 Child processe are over out of 100
in thread Parallel::ForkManager How to wait when 1st 10 Child processe are over out of 100

You are mostly right, in the end he does want to run 1000 jobs, but only have 100 at a time max and in blocks of 10.

When calling P::FM you declare how many forks you want to run. In the main method:

my $max_children = 100; my $job_size = 10; my $main_pfm = P::FM->new( $max_children/$job_size );

So that is 100/10 which is 10 job blocks at the same time. Since the job size is 10, then that is 100 jobs at the same time.

my $total_children = 1000; my $job_size = 10; foreach ( 1..$total_children/$job_size ) {

So then we loop through our data (F::PM blocks if there is no avalible slots) 1000/10 is 100. So the main method loops through 100 job blocks while only running 10 blocks at a time.

  • Comment on Re^3: Parallel::ForkManager How to wait when 1st 10 Child processe are over out of 100
  • Select or Download Code

Replies are listed 'Best First'.
Re^4: Parallel::ForkManager How to wait when 1st 10 Child processe are over out of 100
by anshumangoyal (Scribe) on Dec 01, 2011 at 18:51 UTC
    I have read a limitation of P:FM that you cannot define new P:FM until all Child processes from the previous are finished. This limitation is mentioned in CPAN for P::FM. So can any one give another suggestions?

      Almost:

      You are free to use a new copy of Parallel::ForkManager in the child processes, although I don't think it makes sense.

      What you saw was:

      If you want to use more than one copies of the Parallel::ForkManager, then you have to make sure that all children processes are terminated, before you use the second object in the main program.

      However, I would test it to double check. Using the run_on_finish callback would work. Whether it would be more clear or easier to read is up to the user.

      You can't define another P::FM object within the same process (until all child processes from the previous are terminated...), but you can define a new object in a child process.