I have always wanted a Barrier module to complement MCE::Child, MCE::Hobo, or threads. The module works on Unix, Cygwin, including Windows. Tonight, I enhanced MCE::Simple to support the spawn loop syntax.
# create the worker processes spawn task($barrier, $_) for 1..5;
Which is transposed to the following. The ident argument defaults to blank '' if omitted.
MCE::Simple::_reap_joinable(); do { MCE::Simple::_async_i('', \&task, $barrier, $_) } for 1..5;
At this moment, the code is 100% completed in MCE::Barrier, MCE::Semaphore, and MCE::Simple. It's a relief. I tested MCE::Barrier on Linux with 4,000 workers, trying to break it :). All the one's come first, the two's, followed by three's and so on. Workers wait for all parties to reach the barrier.
use strict; use warnings; use MCE; use MCE::Barrier; use Time::HiRes qw(time); my $num_workers = 4000; my $barrier = MCE::Barrier->new($num_workers); my $start = time(); MCE->new( max_workers => $num_workers, user_func => sub { my $id = MCE->wid; for (1..400) { MCE->say("$_: $id"); $barrier->wait; } } )->run(); printf {*STDERR} "\nduration: %0.3f\n\n", time() - $start;
The sync implementation in MCE is two stages; sync and unsync. MCE::Barrier's implementation is one stage with alternating semaphores (1 or 2) k = 3 - k. A limiter prevents many workers attempting read IO concurrently or simultaneously. The result is a fast implementation no matter the number of parties.
In reply to Re^2: Code brewing for the upcoming MCE 10 year anniversary
by marioroy
in thread Code brewing for the upcoming MCE 10 year anniversary
by marioroy
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |