in reply to MCE - max_workers

Hi Ken,

That's a great idea. I can add a regex check for %; i.e. max_workers => '25%' and set the value accordingly. It will clamp to 1 minimum and MCE::Util::get_ncpu() maximum.

With regards to max_workers => 'auto', that limits to 8 max including on machines with more than 8 logical cores. It was a safety mechanism at the time to not have MCE consume more cores for heavy IO related tasks. Basically, I did not want for MCE to be the reason why a server went down.

auto*0 is not a bug. The minimum number of workers is 1.

Replies are listed 'Best First'.
Re^2: MCE - max_workers
by kcott (Archbishop) on Nov 16, 2021 at 04:43 UTC

    G'day Mario,

    "That's a great idea."

    Thanks for the positive feedback.

    "I can add a regex check for % ..."

    That would be great. Having it in the back-end (e.g. an MCE class):

    use MCE; ... max_workers => "$percent%"

    rather than in an intermediary module (e.g. a Kens::MCE::Util class):

    use MCE; use Kens::MCE::Util 'get_max_workers'; ... max_workers => get_max_workers($percent)

    would certainly simplify matters.

    "auto*0 is not a bug."

    Thanks for the clarification.

    — Ken

      Hi Ken,

      MCE 1.875 was released moments ago. Folks may specify a percentage for max_workers. I enjoyed writing the MCE::Flow test in 03_max_workers.t.

      use strict; use warnings; use Test::More; BEGIN { use_ok 'MCE'; use_ok 'MCE::Flow'; } { no warnings 'redefine'; sub MCE::Util::get_ncpu { return 16; } } # ... lots more tests not shown here { MCE::Flow::init(max_workers => [1, '25%']); my @res; mce_flow { gather => \@res }, sub { MCE->gather('a'.MCE->task_wid()); }, # 1 worker sub { MCE->gather('b'.MCE->task_wid()); }; # 4 workers @res = sort @res; is("@res", "a1 b1 b2 b3 b4", "check that MCE::Flow ran with 5 worke +rs"); MCE::Flow->finish(); } done_testing;

      Thanks,

        G'day Mario,

        ++ Thanks for the incredibly fast response and provision of an updated version with this percentage enhancement. I'm only sorry that I'm not allowed to upvote this more than once.

        I've updated my version locally (Cygwin; Perl 5.34) without any issues at all:

        $ cpan ... cpan[1]> install MCE ... MARIOROY/MCE-1.875.tar.gz /usr/bin/make install -- OK

        [That'll you get you a green patch on the testers matrix when it's next updated.]

        Again, many thanks. I'm very impressed with your amazingly quick response.

        — Ken