in reply to Re: MCE - max_workers
in thread MCE - max_workers

G'day Karl,

Let's say I have three clients (users of my MCE application): A has 4 lcores; B has 8 lcores; C has 16 lcores.

Let's further say that each client only wants to use half of their processing power for my application, so: A wants to use 2 lcores; B wants to use 4 lcores; C wants to use 8 lcores.

If I simply use an "int param", as you suggest, they'll get the same value for max_workers. For instance, if I used max_workers => 6: A would be using 150%; B would be using 75%; C would be using 37.5%. (Compare what happened, in the first half of my OP, where I didn't cap the value and, at 200%, my computer with 12 lcores got 24 workers.)

Furthermore, clients' requirements may change depending on a number of factors. Perhaps start a big job on a Friday afternoon, using all available processing power, and let it run over the weekend; in another scenario, run jobs mid-week, during a busy part of the day, using only a fraction of available processing power.

Both of my solutions calculate the percentages correctly. What I didn't know was if there was a better solution; and, if not, which of those two solutions fellow monks would consider to be better (keeping in mind, of course, that better might vary under different conditions).

— Ken

Replies are listed 'Best First'.
Re^3: MCE - max_workers
by karlgoethebier (Abbot) on Jul 01, 2021 at 19:36 UTC

    May be that I still miss the point. But it is like you go to your favorite restaurant: It has n cooks. As a guest AKA client you can’t influence this number to get your diner faster. You have to wait. You get in a queue. Same thing as if you take this Magic Bus

    «The Crux of the Biscuit is the Apostrophe»