Now, I wonder about Python vs Perl. And then, something wonderful...
Python
from sys import stdout
from multiprocessing import Process
from multiprocessing import Barrier
def task(barrier, number):
for i in range(1, 400 + 1):
print(f'{i}: {number}')
barrier.wait()
if __name__ == '__main__':
stdout.flush()
num_workers = 500
workers = list()
# create a barrier
barrier = Barrier(num_workers)
# create the worker processes
for i in range(1, num_workers + 1):
workers.append(Process(target=task, args=(barrier, i)))
workers[-1].start()
# wait for all processes to finish
for w in workers:
w.join()
Perl
This seems odd, I know. Folks cannot run this. MCE::Simple will be released soon. I want to share the performance uptake having a limiter.
use MCE::Simple -strict, -signatures;
use MCE::Barrier;
sub task ($barrier, $number) {
for (1 .. 400) {
say "$_: $number";
$barrier->wait;
}
}
sub main () {
STDOUT->autoflush(1);
my $num_workers = 500;
# create a barrier
my $barrier = MCE::Barrier->new($num_workers);
# create the worker processes
spawn task($barrier, $_) for 1 .. $num_workers;
# wait for all processes to finish
$_->join for MCE::Simple->list; # same as sync;
}
main();
Results
Python3 and Perl complete in 4.1 seconds. That was with the limiter commented out. The time is 1.5 seconds with the limiter enabled.
sub new {
...
# [10] limiter
(!$is_winenv && !$tid) ? MCE::Semaphore->new(3,1) : (),
...
}
sub _wait {
...
$obj->[10]->down if $obj->[10]; # limiter
my ($parties, $synced, $timed_out, $aborted) = $obj->_up($flag);
$obj->[10]->up if $obj->[10];
...
}
The other oddity is that I cannot spin 1,000 processes in Python. It complains about running out of file handles. Perl, no problem!