Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question: 

use strict;
use warnings;
use feature 'say';

use MCE::Map;

MCE::Map-> init(
    chunk_size => 1, 
    max_workers => 4,
    user_begin => sub {
        print "## ", MCE-> wid, " started\n";
    },
    user_end => sub {
        print "## ", MCE-> wid, " completed\n";
    },
);

my @a = MCE::Map-> run( sub { $_ }, [ 0 .. 9 ]);
print "\n", "@a", "\n";

say $MCE::VERSION;

[download]

This, of course, prints:

## 1 started
## 3 started
## 2 started
## 4 started
## 1 completed
## 3 completed
## 2 completed
## 4 completed

0 1 2 3 4 5 6 7 8 9
1.882

[download]

I want to make dependency on MCE, in my module, very optional i.e. triggered by rare circumstances only. Replacing "use" with "require" results in just first 4 lines of output and process running indefinitely, 4 cores fully loaded.

MCE::Map being just a convenience, I can do this instead:

use strict;
use warnings;
use feature 'say';

require MCE;

my @in = ( 0 .. 9 );
my @out;

my $mce = MCE-> new(
    chunk_size => 1, 
    max_workers => 4,
    user_begin => sub {
        print "## ", MCE-> wid, " started\n";
    },
    user_end => sub {
        print "## ", MCE-> wid, " completed\n";
    },
    user_func => sub { MCE-> gather( $_ )},
    input_data => \@in,
    gather => \@out,
);

$mce-> run;
say "@out";
$mce-> shutdown;

[download]

And it runs perfectly. I understand I'll have to deal now with ordering of the output, it's not an issue. But that's the reason I was hoping to use MCE::Map to make it easier. Is this a bug?

In general, is very optional (documented) dependency on something big and complex as MCE a bad-bad-bad idea? I guess I'll have to indicate minimum required MCE version then, which for example above would be... 1.500?

Replies are listed 'Best First'.
Re: Can't require MCE::Map (Strawberry)?
by Discipulus (Canon) on Mar 07, 2025 at 09:18 UTC
    Hello Anonymous Monk (consider registering an account ;),

    Unfortunately not a MCE expert, just a big promoter, but generally speaking use X is transalted with require X; import X LIST eventually inside BEGIN to mimicry exatly use

    Consider, in StrawberryPerl: 

    C>perl -we "require MCE::Map; MCE::Map-> init(user_begin => sub {},use
+r_end => sub {}); my @a = MCE::Map->run( sub { $_ }, [ 0 .. 9 ]);"
# CTRL-C
Terminating on signal SIGINT(2)
MCE::shutdown: method is not allowed while running at -e line 0.
END failed--call queue aborted, <__ANONIO__> line 80992.

                             # LOOK HERE    #
C>perl -we "require MCE::Map; import MCE::Map; MCE::Map-> init(user_be
+gin => sub {},user_end => sub {}); my @a = MCE::Map->run( sub { $_ },
+ [ 0 .. 9 ]);"

C>

    [download]

    L*

    There are no rules, there are no thumbs..
    Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.
[reply]
[d/l]
[select]

      Yeah, thanks. Apparently, MCE::Map::import pre-assigns 'auto' to the CHUNK_SIZE property of a global, and if it's not assigned, then because of line 235, just before 'run', the chunk size is returned/gets undefined (why?) and somehow silently treated as zero (just guessing, didn't debug further), hence infinite loop. I'm now inclined to using the MCE proper anyway, but I think the situation is worth for Mario to have a look at.

[reply]
[d/l]
[select]
        Have you considered filing an actual bug report? Or tagging Mario on this, which will now generate a notification for him?
[reply]
Re: Can't require MCE::Map (Strawberry)?
by marioroy (Prior) on Mar 26, 2025 at 21:51 UTC

    Not forget the immediate import statement if calling require, necessary for the MCE Models.

    use strict;
use warnings;
use feature 'say';

require MCE::Map;
import MCE::Map;

MCE::Map-> init(
    chunk_size => 1, 
    max_workers => 4,
    user_begin => sub {
        print "## ", MCE-> wid, " started\n";
    },
    user_end => sub {
        print "## ", MCE-> wid, " completed\n";
    },
);

my @a = MCE::Map-> run( sub { $_ }, [ 0 .. 9 ]);
MCE::Map-> finish;
print "\n", "@a", "\n";

say $MCE::VERSION;

__END__
## 1 started
## 3 started
## 2 started
## 4 started
## 2 completed
## 4 completed
## 1 completed
## 3 completed

0 1 2 3 4 5 6 7 8 9
1.901

    [download]

    Here is the MCE variant. Ordered output is possible using MCE::Candy.
    Note $chunk_id as the first argument to gather. MCE->chunk_id works too.

    use strict;
use warnings;
use feature 'say';

require MCE;
require MCE::Candy;

my @in = ( 0 .. 9 );
my @out;

my $mce = MCE-> new(
    chunk_size => 1,
    max_workers => 4,
    user_begin => sub {
        print "## ", MCE-> wid, " started\n";
    },
    user_end => sub {
        print "## ", MCE-> wid, " completed\n";
    },
    user_func => sub {
        my ( $mce, $chunk_ref, $chunk_id ) = @_;
        MCE-> gather( $chunk_id, $_ )
    },
    input_data => \@in,
    gather => MCE::Candy::out_iter_array(\@out),
);

$mce-> run;
$mce-> shutdown;
say "@out";

__END__
## 1 started
## 2 started
## 4 started
## 3 started
## 3 completed
## 4 completed
## 2 completed
## 1 completed
0 1 2 3 4 5 6 7 8 9

    [download]
[reply]
[d/l]
[select]
Re: Can't require MCE::Map (Strawberry)?
by marioroy (Prior) on Mar 26, 2025 at 20:27 UTC

    Hi. I missed seeing this and will report back. Thank you, the monk who messaged me. It's been a while since I logged into PerlMonks.

[reply]

Back to Seekers of Perl Wisdom