Re^2: Perl threads loss of performance with system call

Please find 2 examples, based on karlgoethebier's demonstration.

Example 1:

The posix_exit Hobo option is beneficial on Unix platforms. Be sure to close any file handles opened by the worker. The void_context option is helpful to reduce IPC overhead if not wanting result from workers.

#!/usr/bin/env perl

use strict;
use warnings;
use MCE::Hobo;
use feature qw(say);
use constant AMOUNT => 0.001;

my $cores = MCE::Util::get_ncpu();

MCE::Hobo->init(
    max_workers => $cores,
    void_context => 1,
    posix_exit => 1,
);

sub consumer_task {
    my $id = shift;
    say qq($id ) . MCE::Hobo->pid();
    qx( sleep @{[ AMOUNT ]} ); 
}

for my $id ( 1..800 ) {
    my $hobo = MCE::Hobo->create( \&consumer_task, $id );
}

MCE::Hobo->wait_all();
[download]

Example 2:

Here, we minimize the spawning of Hobo workers and instead use a channel or a queue to send/receive input.

#!/usr/bin/env perl

use strict;
use warnings;
use MCE::Hobo;
use MCE::Channel;
use feature qw(say);
use constant AMOUNT => 0.001;

my $cores = MCE::Util::get_ncpu();
my $queue = MCE::Channel->new();

MCE::Hobo->init(
    void_context => 1,
    posix_exit => 1,
);

# Consumers

sub consumer_task {
    while ( my $input = $queue->dequeue() ) {
        say qq($input->{id} ) . MCE::Hobo->pid();
        qx( sleep @{[ AMOUNT ]};) 
    }
}

MCE::Hobo->create( \&consumer_task ) for 1..$cores;

# Producer

for ( 1..800 ) {
    $queue->enqueue({ id => $_ });
}

$queue->end();

MCE::Hobo->wait_all();
[download]

Comment on Re^2: Perl threads loss of performance with system call Select or Download Code

Replies are listed 'Best First'.
Re^3: Perl threads loss of performance with system call by Anonymous Monk on Jul 14, 2021 at 06:09 UTC
The channel object is bidirectional. Therefore, the worker can send its PID value to the producer. This has the effect of max one item in the channel. The producer writes to STDOUT like the demonstration by karlgoethebier. #!/usr/bin/env perl use strict; use warnings; use MCE::Hobo; use MCE::Channel; use feature qw(say); use constant AMOUNT => 0.001; my $cores = MCE::Util::get_ncpu(); my $queue = MCE::Channel->new(); MCE::Hobo->init( void_context => 1, posix_exit => 1, ); # Consumers sub consumer_task { while ( my $input = $queue->recv() ) { my $id = $input->{id}; $queue->send2( MCE::Hobo->pid() ); qx( sleep @{[ AMOUNT ]};) } } MCE::Hobo->create( \&consumer_task ) for 1..$cores; # Producer for my $id ( 1..800 ) { $queue->send({ id => $id }); my $hobo_pid = $queue->recv2(); say qq($id ) . $hobo_pid; } $queue->end(); MCE::Hobo->wait_all(); [download]	[reply] [d/l]
Re^3: Perl threads loss of performance with system call by karlgoethebier (Abbot) on Jul 14, 2021 at 10:00 UTC
Thanks. This looks good. What I never understood is why to call `MCE::Hobo->wait_all();` in this context. As it seems that all the hobos terminate correctly in this ~~fubar~~ loop without the call. ŤThe Crux of the Biscuit is the Apostropheť	[reply] [d/l]