in reply to Re: Parallel::ForkManager How to wait when 1st 10 Child processe are over out of 100
in thread Parallel::ForkManager How to wait when 1st 10 Child processe are over out of 100

I believe he wants 100 jobs total to run simultaneously, so if the parent runs 10 processes and each child runs 10, then there will be 100 total. You currently have the parent running 100...I don't think he wants 1000 processes.

Update: my mistake..this is not the post you are looking for...move along

  • Comment on Re^2: Parallel::ForkManager How to wait when 1st 10 Child processe are over out of 100

Replies are listed 'Best First'.
Re^3: Parallel::ForkManager How to wait when 1st 10 Child processe are over out of 100
by admiral_grinder (Pilgrim) on Dec 01, 2011 at 17:02 UTC

    You are mostly right, in the end he does want to run 1000 jobs, but only have 100 at a time max and in blocks of 10.

    When calling P::FM you declare how many forks you want to run. In the main method:

    my $max_children = 100; my $job_size = 10; my $main_pfm = P::FM->new( $max_children/$job_size );

    So that is 100/10 which is 10 job blocks at the same time. Since the job size is 10, then that is 100 jobs at the same time.

    my $total_children = 1000; my $job_size = 10; foreach ( 1..$total_children/$job_size ) {

    So then we loop through our data (F::PM blocks if there is no avalible slots) 1000/10 is 100. So the main method loops through 100 job blocks while only running 10 blocks at a time.

      I have read a limitation of P:FM that you cannot define new P:FM until all Child processes from the previous are finished. This limitation is mentioned in CPAN for P::FM. So can any one give another suggestions?

        Almost:

        You are free to use a new copy of Parallel::ForkManager in the child processes, although I don't think it makes sense.

        What you saw was:

        If you want to use more than one copies of the Parallel::ForkManager, then you have to make sure that all children processes are terminated, before you use the second object in the main program.

        However, I would test it to double check. Using the run_on_finish callback would work. Whether it would be more clear or easier to read is up to the user.

        You can't define another P::FM object within the same process (until all child processes from the previous are terminated...), but you can define a new object in a child process.