G'day All,

I haven't use MCE previously and decided to get my head around it. I believe some of my current personal projects could benefit from it; there are also some $work applications.

I ran the very first example code in "MCE::Core - SYNOPSIS" and that worked fine.

[Note: In the following, "lcores" refers to "logical CPU cores"; MCE doco uses the same abbreviation.]

I decided that it would be useful to allow users to specify what percentage of lcores were used by an application. For instance, at any given time, 50% might be allowed for one MCE process, 25% for another MCE process, leaving the remaining 25% for other general use on the computer.

My first thought was that "max_workers => 'auto'" was unsuitable for this; even with its arithmetic modifiers (see "MCE::Core - MCE->new()" and "MCE::Util - get_ncpu()" for more about that). It uses an arbitrary eight or all lcores, whichever is the smallest: not useful for determining a percentage of lcores. So, I came up with the following proof-of-concept (POC) code:

#!/usr/bin/env perl

use 5.016;
use warnings;

use MCE;

my @percentages = qw{0 20 25 33.333 40 50 60 66.667 75 80 100 200};

for my $specified_percentage (@percentages) {
    say "Specified percentage of logical cores to use: $specified_perc
+entage%";
    my $modified_percentage
        = $specified_percentage <= 0   ? 1
        : $specified_percentage >= 100 ? 100
        :                                $specified_percentage;
    say "Modified percentage of logical cores to use: $modified_percen
+tage%";
    say 'Total number of logical cores ', MCE::Util::get_ncpu();
    my $max_workers
        = int(0.5 + MCE::Util::get_ncpu() * $modified_percentage / 100
+) || 1;
    say "Maximum workers to use: $max_workers";

    my @wids;

    my $mce = MCE::->new(
        max_workers => $max_workers,
        gather => \@wids,
        user_func => sub {
            my ($mce) = @_;
            MCE->gather($mce->wid);
        },
    );

    $mce->run();

    say "Worker ID(s): @{[sort { $a <=> $b } @wids]}";
}
[download]

[Note: The use 5.016 is $work related: the minimum version to use. 5.012 will cover strict and say; MCE only requires 5.008001.]

The output from this looked fine. I've put it inside the spoiler.

The percentage needed to be capped at 100. If I comment out ": $specified_percentage >= 100 ? 100", the last block of output, which was

Specified percentage of logical cores to use: 200%
Modified percentage of logical cores to use: 100%
Total number of logical cores 12
Maximum workers to use: 12
Worker ID(s): 1 2 3 4 5 6 7 8 9 10 11 12
[download]

becomes

Specified percentage of logical cores to use: 200%
Modified percentage of logical cores to use: 200%
Total number of logical cores 12
Maximum workers to use: 24
Worker ID(s): 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
+ 23 24
[download]

I could put a version of that code into a module with a function returning $max_workers.

I decided to revisit 'auto' and see if I could come up with something better. Here's that POC solution:

#!/usr/bin/env perl

use 5.016;
use warnings;

use MCE;

my $total_logical_cores = MCE::Util::get_ncpu();
my $auto_max_workers;

{
    my $mce_temp = MCE::->new(max_workers => 'auto');
    $auto_max_workers = $mce_temp->max_workers();
    $mce_temp->shutdown;
}

my $auto_to_ncpu_mult = $total_logical_cores / $auto_max_workers;
my @percentages = qw{0 20 25 33.333 40 50 60 66.667 75 80 100 200};

for my $percentage (@percentages) {
    say "Specified percentage of logical cores to use: $percentage%";
    $percentage = 1 if $percentage <= 0;
    say "Modified percentage of logical cores to use: $percentage%";
    say 'Total number of logical cores ', MCE::Util::get_ncpu();
    my $auto_max_workers = 'auto*' . $auto_to_ncpu_mult * $percentage 
+/ 100;
    say "Auto max_workers: $auto_max_workers";

    my @wids;

    my $mce = MCE::->new(
        max_workers => $auto_max_workers,
        gather => \@wids,
        user_func => sub {
            my ($mce) = @_;
            MCE->gather($mce->wid);
        },
    );

    $mce->run();

    say "Worker ID(s): @{[sort { $a <=> $b } @wids]}";
}
[download]

The output looks fine; again, it's in the spoiler.

Because 'auto' caps the maximum number of workers at the maximum number of lcores, I didn't need to do anything special with that. Here's the last block of output for comparison with what I showed above.

Specified percentage of logical cores to use: 200%
Modified percentage of logical cores to use: 200%
Total number of logical cores 12
Auto max_workers: auto*3
Worker ID(s): 1 2 3 4 5 6 7 8 9 10 11 12
[download]

However, I did need to set the minimum percentage. If I comment out "$percentage = 1 if $percentage <= 0;, the first block of output, which was

Specified percentage of logical cores to use: 0%
Modified percentage of logical cores to use: 1%
Total number of logical cores 12
Auto max_workers: auto*0.015
Worker ID(s): 1
[download]

becomes

Specified percentage of logical cores to use: 0%
Modified percentage of logical cores to use: 0%
Total number of logical cores 12
Auto max_workers: auto*0
Worker ID(s): 1 2 3 4 5 6 7 8
[download]

Curious: 'auto*0' eq 'auto*1' — is that a bug?

I could also put a version of that code into a module with a function returning $auto_max_workers.

So, finally the questions :-)

• Have I made any stupid, novice mistakes?
• Is there a better way to do this?
• Assuming there's no better way, what are the pros and cons of the solutions that I've presented?

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.