in reply to Re: Perl forked processes and variable sharing
in thread Perl forked processes and variable sharing
I met to add that MCE::Hobo and MCE::Shared provide capabilities similar to threads and threads::shared respectively. One isn't required to use MCE workers. In fact, MCE::Shared works also with threads, Parallel::ForkManager, and workers spawned with fork.
If using non-MCE workers, then call MCE::Shared->init inside the child. That will spread IPC communication across 10 data channels. The channel assignment is done automatically for threads and MCE workers (i.e. MCE, MCE::Child, and MCE::Hobo).
unless (fork) { # This assigns the worker 1 of 10 data channels in a # round-robin fashion for the life of the worker. MCE::Shared->init(); do_subroutine (); exit; }
Update: Added demonstrations.
Below, please find two demonstrations which one may run for comparison. Parallel data channels are helpful especially when a reply is not needed i.e. incrementing a value or writing to a shared output object. Each worker increments the value 100,000 times.
use strict; use warnings; use MCE::Shared; use Time::HiRes 'time'; my $number = MCE::Shared->scalar(0); sub do_subroutine { $number->incr for 1..1e5; } my @pids; my $start = time; for (1..10) { my $pid; unless ($pid = fork) { do_subroutine (); exit; } push @pids, $pid if defined $pid; } waitpid $_, 0 for @pids; printf "number %d\n", $number->get; printf "seconds %.03f\n", time - $start; __END__ number 1000000 seconds 5.234
That's quite fast considering incrementing the value involves IPC. Faster is possible if you want non-MCE workers to reach levels capable of threads and MCE workers which call MCE::Shared->init automatically.
MCE::Shared->init
Calling MCE::Shared->init is beneficial for non-MCE workers.use strict; use warnings; use MCE::Shared; use Time::HiRes 'time'; my $number = MCE::Shared->scalar(0); sub do_subroutine { $number->incr for 1..1e5; } my @pids; my $start = time; for (1..10) { my $pid; unless ($pid = fork) { MCE::Shared->init; # enables multi-channel do_subroutine (); exit; } push @pids, $pid if defined $pid; } waitpid $_, 0 for @pids; printf "number %d\n", $number->get; printf "seconds %.03f\n", time - $start; __END__ number 1000000 seconds 2.086
That is shy of 500k per second on Clear Linux.
threads demonstration
For completeness adding threads and MCE::Hobo demonstrations. They benefit from multi-channel IPC automatically and not necessary to call MCE::Shared->init.
use strict; use warnings; use threads; use MCE::Shared; use Time::HiRes 'time'; my $number = MCE::Shared->scalar(0); my $start = time; sub do_subroutine { $number->incr for 1..1e5; } threads->create('do_subroutine') for 1..10; $_->join for threads->list; printf "number %d\n", $number->get; printf "seconds %.03f\n", time - $start; __END__ number 1000000 seconds 2.142
MCE::Hobo demonstration
use strict; use warnings; use MCE::Hobo; use MCE::Shared; use Time::HiRes 'time'; my $number = MCE::Shared->scalar(0); my $start = time; sub do_subroutine { $number->incr for 1..1e5; } MCE::Hobo->create('do_subroutine') for 1..10; $_->join for MCE::Hobo->list; # same as MCE::Hobo->wait_all printf "number %d\n", $number->get; printf "seconds %.03f\n", time - $start; __END__ number 1000000 seconds 2.087
|
|---|