in reply to Re: Return all the data from child to parent with Parallel::Forkmanager
in thread Return all the data from child to parent with Parallel::Forkmanager
Hi again,
The following provides a MCE::Loop demonstration. This time, am using a hash for storing the returned data. Workers pass data using socket handles inside MCE. Thus, no temp files are made for sending data to the parent process.
use strict; use warnings; use feature 'say'; use MCE::Loop; my $threads = 4; my @name = ('read_genome', 'read_mapfile', 'read_GTF', 'read_RM'); my @task = (\&read_genome, \&read_mapfile, \&read_GTF, \&read_RM); my %ret; if ($threads == 1) { for my $id (0 .. $#task) { $ret{"$id"} = eval { $task[$id]->() }; $ret{"$id:err"} = $@; } } else { MCE::Loop->init( max_workers => $threads, posix_exit => 1, chunk_size => 1 ); %ret = mce_loop { my $id = $_; my $res = eval { $task[$id]->() }; MCE->gather( $id => $res, "$id:err" => $@ ); } [ 0 .. $#task ]; MCE::Loop->finish; } sub read_genome { # do something return { 'aa' => 'bb' }; } sub read_mapfile { # do something return { 'cc' => 'dd' }; } sub read_GTF { # do something die 'exception'; return { 'ee' => 'ff' }; # not reached } sub read_RM { # do something return { 'gg' => 'hh' }; } # use data generated in the subroutines use Data::Dumper; for my $id (0 .. $#task) { say "## ", $name[$id]; if (length $ret{"$id:err"}) { say "ERROR: ", $ret{"$id:err"}; next; } say Dumper($ret{$id}); }
Output:
## read_genome $VAR1 = { 'aa' => 'bb' }; ## read_mapfile $VAR1 = { 'cc' => 'dd' }; ## read_GTF ERROR: exception at j2.pl line 45, <__ANONIO__> line 2. ## read_RM $VAR1 = { 'gg' => 'hh' };
Regards, Mario
|
|---|