in reply to Forking problem UPDATED

I was going to propose the parent populate the array, create a pipe, create N children, and do print PIPE "$_\n" for 0..$#array; so each child could read (atomically) the index to the next data to be processed.

But it looks like Parallel::ForkManager already pretty much does that for you so perhaps you should just use that.

- tye        

Replies are listed 'Best First'.
Re^2: Forking problem (manager)
by avi_learns_perl (Novice) on Mar 06, 2007 at 17:34 UTC
    i thought abt Parallel::ForkManager but my problem is that i need the pipes to be recreated dynamically such that after each child finishes there should be another one created...
[reply]

      after each child finishes there should be another one created

      That's exactly what Parallel::ForkManager does.

      Say you have 5 tasks, and you set a max of 3 children. ForkManager will create 3 children and assign them tasks 0, 1 and 2. When one of these end, another child is created for task 3. When one of child ends, another child is created for task 4. Having no more tasks to assign, it waits for the 3 children to end.

      The implementation is very simple: 

      use Parallel::ForkManager qw( )

use constant MAX_PROCESSES => 3;

my $pm = Parallel::ForkManager->new(MAX_PROCESSES);
foreach $data (@array) {
   # Forks and returns the pid for the child:
   my $pid = $pm->start and next; 

   ... do some work with $data in the child process ...

   $pm->finish; # Terminates the child process
}

      [download]
[reply]
[d/l]
        yes...this is when each time the fork does the same job with different $ data....but i have 3 different jobs using $data from array such that 3 run simultaneously... example for $data1 I do print $data1 for $data2 i do system "/home/working/perl1.pl ".$data2; for $data3 I do system "/home/perl2.pl ".$data3 all three have to run simultaneously and when any one ends $data4 has to be sent as input to that program...example assuming $data2 finishes the fork should then execute the system"/home/working/perl1.pl ".$data4;...this goes on as you say till all data in @array is exhausted
[reply]

In Section Seekers of Perl Wisdom