in reply to Background forked processes from a function.
It sounds to me like you need something like Parallel::ForkManager to handle your many sub processes, each doing a query. That module will keep track of all the children for you, and make it easy to write the code to merge the query results from all the queries in those children.
Something like this perhaps:
#!/usr/bin/perl use Parallel::ForkManager; my $max_threads = 5; # Chose a number for best performace. my $fork_mgr = new Parallel::ForkManager($max_threads); my @big_list = ( { query => 'SELECT foo FROM tblBar WHERE baz LIKE `%frob%`', started => 0, complete=> 0, exit_code=> undef, results => [], }, # etc ); $fork_mgr->run_on_finish ( sub { my($child_pid, $exit_code, $child_id, $exit_signal, $core_dump +, $ret_data) = @_; # Code to store the results from each thread go here. # eg: $big_list[$child_id]{exit_code} = $exit_signal; $big_list[$child_id]{complete} = 1; $big_list[$child_id]{results} = $ret_data; } ); for( my $itemNum=0; $itemNum<scalar @big_list; $itemNum++ ) { $fork_mgr->start($itemNum) and next URL # Code in this block will run in parallel my $thisQuery = $big_list[$itemNum]; my $result = do_query($thisQuery{query}); if( $result->ran_OK() ) { $thisQuery{results} = $result->get_results(); } # Store the final result. The value you pass to finish, # and the data structure reference will be # received by the sub you defined in run_on_finish $fork_mgr->finish( $result->ran_OK(), $thisQuery{results} ); } $fork_mgr->wait_all_children(); # Now all child threads have finished, # your results should be available.
|
|---|