in reply to Re^4: help with Parallel:ForkManager
in thread help with Parallel:ForkManager

You are a man of few words!

You said it was different than fork(), which meant you knew how fork() worked. I pointed out that it isn't different than fork().

Is that about right?

Up until the last sentence, only minor errors or clarifications need to be made:

When I do '$pm->start and next', it forks a child which takes the first argument of the @command arraygets a copy of the current process's memory space including $command and starts executing. At the same time, without waiting for the 1st child to finish, itthe parent forks another child that processes the 2nd argument of the array and so on, until 20 children are running in parallel. If 20 children already running, but there are still unprocessed elements in the array, it would wait until a child would finish and forkbefore forking another child.

But then you followed with:

The array is processed by children only, the parent doesn't go through the array.

It might depend on what you mean by "processed" and "go through", but that sounds completely backwards.

You just described how the parent forks a child for each array element and how each child processes only one element of the array.

Code executed in parent:

for my $command (@commands) { $manager->start and next; }

Code executed in child:

0 and next; exec( $command ); $manager->finish;

Note the lack of array in the code executed by the child?

Replies are listed 'Best First'.
Re^6: help with Parallel:ForkManager
by arthurs (Novice) on Jan 13, 2010 at 20:44 UTC
    Thanks. Now I really understand. But there is another question if I may. Suppose that instead of @command array, I have a list (array) of files, @files_list. In a loop, I open a file from the list, read through it, set some counters, close the file, then get the 2nd file from the list, do the same thing and so on. The processing of each file takes time, and I have a lot of files. So I use the parallel fork to start four children ($manager->start and next) where each child would process a file from a list. Between’ $manager->start and next’ and $manager->finish, I set some counters, say how many files could not be opened. That counter is not global to all the children, so my counter would not be accurate. My question is what do I do if I need a variable to be shared (or global) between all the children, so each child would be able to update the same variable. I don't want to store the counter in some temp file that I would be able to access after all children are finished. Thanks much.

      You are looking to do interprocess communication. It's a very lengthy topic.

      It might be simpler to use threads (or forks's thread-like interface). You can create a pool of workers that take jobs from a queue. BrowserUk has made numerous posts on the subject.

      They can report their status back via another queue (or via a shared variable if we're talking about counter).

        OK, thanks for your help. I will look into this subject. Thanks.