in reply to Code brewing for the upcoming MCE 10 year anniversary

Behind the scene, I'm using MCE (chunk_size set to 1). 
mce_foreach my $var ( @list ) { ... }
mce_foreach ( @list ) { ... }

mce_foreach_f my $var ( $path or $fh ) { ... }
mce_foreach_f ( $path or $fh ) { ... }

mce_foreach_s my $var ( @list ) { ... }
mce_foreach_s ( @list ) { ... }

[download]

The best part is the spawn and sync keywords.

# context scalar, array, or hash

spawn (my|our)* $var @var or %var = sub { codeblock };
spawn (my|our)* $var @var or %var = sub { func(@_) }, args;
spawn (my|our)* $var @var or %var = func(arg1, ...);
spawn (my|our)* $var @var or %var = func();

sync $var;
sync @var;
sync %var;

# anonymous code blocks

spawn $identVar or "quotedString", sub { codeblock };
spawn sub { codeblock };

sync;

[download]

Replies are listed 'Best First'.
Re^2: Code brewing for the upcoming MCE 10 year anniversary
by marioroy (Prior) on Oct 23, 2022 at 14:25 UTC

    Sometimes, the use-case calls for processing at the chunk level inside the code block. For example, the worker receives the chunk as a whole.

    mce_forchunk
mce_forchunk_f
mce_forchunk_s

    [download]

    I will use MCE::Loop behind the scene.

[reply]
[d/l]

      The mce_forchunk keywords are working. Sometimes, it's preferably to process at the chunk level. So, here it is.

      mce_forchunk

      use MCE::Simple -strict, max_workers => 4, init_relay => '';

mce_forchunk (11..19) {
    my ($mce, $chunk_ref, $chunk_id) = @_;
    for my $i (@{ $chunk_ref }) {
        MCE->say($i);
    }
}

mce_forchunk (31..39) {
    for my $i (@{ $_ }) {
        MCE->say($i);
    }
}

mce_forchunk my $chunk (51..59) {
    for my $i (@{ $chunk }) {
        MCE->say($i);
    }
}

mce_forchunk my $chunk (71..79) {
    my $output = "From chunk_id: ".MCE->chunk_id."\n";
    for my $i (@{ $chunk }) {
        $output .= "$i\n";
    }
    MCE::relay {
        # write directly to STDOUT, not involve manager process
        print $output;
    };
}

      [download]

      Output

      15
13
11
16
14
17
12
18
19
33
34
31
39
32
37
35
38
36
51
57
53
52
55
58
54
56
59
From chunk_id: 1
71
72
From chunk_id: 2
73
74
From chunk_id: 3
75
76
From chunk_id: 4
77
78
From chunk_id: 5
79

      [download]

      Spawn MCE asynchronously

      MCE::Simple expands keywords at compile time to parallel code.

      use MCE::Simple (
    include => [qw/ -strict -signatures /],
    max_workers => 4, init_relay => '',
);

MCE::Simple->init(
    on_finish => sub ( $pid, $exit, $ident, $signal, $error, @ret ) {
        say "MCE job ** $ident ** completed; status $exit.";
        print join('', @ret);
    },
);

spawn "foo", sub {
    my @results;
    MCE::Simple->init(gather => \@results);

    mce_forchunk my $chunk (91..99) {
        my $output = "From chunk_id: ".MCE->chunk_id."\n";
        for my $i (@{ $chunk }) {
            $output .= "$i\n";
        }
        MCE::relay {
            MCE->gather($output);
        };
    }

    @results;
};

# do something else

sync;

      [download]

      Output

      MCE job ** foo ** completed; status 0.
From chunk_id: 1
91
92
From chunk_id: 2
93
94
From chunk_id: 3
95
96
From chunk_id: 4
97
98
From chunk_id: 5
99

      [download]
[reply]
[d/l]
[select]

        Or using MCE::Loop instead. One can have MCE::Loop gather orderly via MCE::relay.

        use MCE::Simple (
    include => [qw/ -strict -signatures -mce_loop /],
    max_workers => 4, init_relay => '',
);

MCE::Simple->init(
    on_finish => sub ( $pid, $exit, $ident, $signal, $error, @ret ) {
        say "MCE job ** $ident ** completed; status $exit.";
        print join('', @ret);
    },
);

spawn "foo", sub {
    my @results = mce_loop {
        my $output = "From chunk_id: ".MCE->chunk_id."\n";
        for my $i (@{ $_ }) {
            $output .= "$i\n";
        }
        MCE::relay {
            MCE->gather($output);
        };
    } 91..99;
};

# do something else

sync;

        [download]

        Thank you, PerlMonks. I was able to think out loud. The demonstrations work flawlessly.

[reply]
[d/l]

In Section Meditations