in reply to Re: Parallel::Forkmanager question
in thread Parallel::Forkmanager question

Hello again jamesgerard1964,

I wanted to try another way using MCE::Loop. It might look something like this.

use strict; use warnings; use MCE::Loop chunk_size => 1, max_workers => 8; my @servers = ('a'..'z'); my $timeout = 10; my $user = "foo"; my $file = "/tmp/hello.txt"; my $rpath = "/tmp"; my $fname = "hello.txt"; mce_loop { my $server = $_; MCE->say("child process running, with a key of $server ($$)"); eval { local $SIG{ALRM} = sub { alarm 0; die "alarm\n" }; alarm $timeout; my $rc = system("scp -p $file $user\@$server:$rpath/$fname"); }; alarm 0; } @servers;

Well, that's another way.

Replies are listed 'Best First'.
Re^3: Parallel::Forkmanager question
by Anonymous Monk on Apr 06, 2017 at 03:00 UTC

    Hello again, jamesgerard1964,

    Fortunately, the MCE::Shared module is not exclusive to MCE workers. The following is a demonstration using Parallel::ForkManager and MCE::Shared. Workers store the status into a shared hash.

    use strict; use warnings; use Parallel::ForkManager; use MCE::Shared; my @servers = ('a'..'z'); my $timeout = 10; my $user = "foo"; my $file = "/tmp/hello.txt"; my $rpath = "/tmp"; my $fname = "hello.txt"; tie my %result, 'MCE::Shared'; my $fork_manager = Parallel::ForkManager->new(8); $fork_manager->set_waitpid_blocking_sleep(0); foreach my $server (@servers) { my $pid = $fork_manager->start; if ($pid) { # parent print "$$: child process started, with a key of $server ($pid)\n"; } else { # child eval { local $SIG{ALRM} = sub { alarm 0; die "alarm\n" }; alarm $timeout; $result{$server} = system("scp -p $file $user\@$server:$rpath/$ +fname"); }; alarm 0; $fork_manager->finish; } } $fork_manager->wait_all_children; for my $server ( sort keys %result ) { my ($status, $signum); $status = $result{$server}; $signum = $status & 127; # extract signal number $status = $status >> 8; # becomes exit status print "$server: exit_status $status, signal_num $signum\n"; }
Re^3: Parallel::Forkmanager question
by Anonymous Monk on Apr 06, 2017 at 02:32 UTC

    Hello again jamesgerard1964,

    One might find useful the exit_status from each system call. The MCE::Shared module is helpful in that regard.

    On purpose, am making a slight change to using MCE::Loop. The comma is needed after the sub { ... } block before @servers. Basically, am calling three methods: ->init to configure MCE options, ->run to run the code block, and finally ->finish to shutdown and reap workers. Afterwards, like to see the system status captured into the shared hash.

    use strict; use warnings; use MCE::Loop; use MCE::Shared; my @servers = ('a'..'z'); my $timeout = 10; my $user = "foo"; my $file = "/tmp/hello.txt"; my $rpath = "/tmp"; my $fname = "hello.txt"; tie my %result, 'MCE::Shared'; MCE::Loop->init( chunk_size => 1, max_workers => 8, ); MCE::Loop->run( sub { my $server = $_; MCE->say("child process running, with a key of $server ($$)"); eval { local $SIG{ALRM} = sub { alarm 0; die "alarm\n" }; alarm $timeout; $result{$server} = system("scp -p $file $user\@$server:$rpath/$fn +ame"); }; alarm 0; }, @servers ); MCE::Loop->finish; for my $server ( sort keys %result ) { my ($status, $signum); $status = $result{$server}; $signum = $status & 127; # extract signal number $status = $status >> 8; # becomes exit status print "$server: exit_status $status, signal_num $signum\n"; }

    Thank you for reaching out to Perlmonks.

      Regarding mce_loop or MCE::Loop->run(...), both do the same thing. The latter requires the comma after the block.

      MCE::Loop->init( ... ); mce_loop { } @data; MCE::Loop->finish; # versus MCE::Loop->init( ... ); MCE::Loop->run( sub { ... }, @data ); MCE::Loop->finish;