in reply to parallelising processes

There was a lengthy thread on this back in november.

In my reply I posted a code example of how to use Parallel::ForkManager. Here it is again:

#!/usr/bin/perl use Parallel::ForkManager; my $max_threads = 8; my $fork_mgr = new Parallel::ForkManager($max_threads); $fork_mgr->run_on_finish ( sub { my($child_pid, $exit_code, $child_id, $exit_signal, $core_dump +) = @_; # Code to store the results from each thread go here. } ); ITEM: foreach my $item (@big_list) { # Fork of child threads. The 'and next' clause will only run # in the parent where start returns the child pid (non zero) $fork_mgr->start($url_ent) and next ITEM; # Code in this block will run in parallel my $result = do_stuff($item); # Store the final result. The value you pass to finish will be # received by the sub you defined in run_on_finish $fork_mgr->finish($result); } $fork_mgr->wait_all_children(); # Now all child threads have finished, # your results should be available.

A few extra tips:

If you are running this code in the perl debugger, then you might want to debug the child threads. If so, run your program in a unix/linux xterm window. The debugger will create new xterm windows for each child thread, so you can separately step through the parent and the children

Conversely, if you don't want to step through the children, and your screen is filling up with windows from child threads, you can use the debug option: o inhibit_exit=0 To suppress the display of windows for child threads that finish without hitting a breakpoint.

Replies are listed 'Best First'.
Re^2: parallelising processes
by RobertCraven (Sexton) on Apr 07, 2011 at 07:58 UTC
    Thanks for your answer, but unfortunately I am not allowed to install Parallel::ForkManager in that environment. Hence to question how to imitate it.
        In theory I could. In practice however, I am not allowed (company policy) to use any additional packages for my projects.