in reply to MCE - max_workers

«…specify what percentage…»

Hi kcott, for the moment I don’t understand why you bother yourself with this percentage stuff. May be that this has something to do with my bad Denglish. AKA that I didn’t understand your problem right. But anyway: Why don’t you hand over the number of CPU‘s as an int param and good is? Best regards, Karl

«The Crux of the Biscuit is the Apostrophe»

Replies are listed 'Best First'.
Re^2: MCE - max_workers
by kcott (Archbishop) on Jul 01, 2021 at 06:41 UTC

    G'day Karl,

    Let's say I have three clients (users of my MCE application): A has 4 lcores; B has 8 lcores; C has 16 lcores.

    Let's further say that each client only wants to use half of their processing power for my application, so: A wants to use 2 lcores; B wants to use 4 lcores; C wants to use 8 lcores.

    If I simply use an "int param", as you suggest, they'll get the same value for max_workers. For instance, if I used max_workers => 6: A would be using 150%; B would be using 75%; C would be using 37.5%. (Compare what happened, in the first half of my OP, where I didn't cap the value and, at 200%, my computer with 12 lcores got 24 workers.)

    Furthermore, clients' requirements may change depending on a number of factors. Perhaps start a big job on a Friday afternoon, using all available processing power, and let it run over the weekend; in another scenario, run jobs mid-week, during a busy part of the day, using only a fraction of available processing power.

    Both of my solutions calculate the percentages correctly. What I didn't know was if there was a better solution; and, if not, which of those two solutions fellow monks would consider to be better (keeping in mind, of course, that better might vary under different conditions).

    — Ken

      May be that I still miss the point. But it is like you go to your favorite restaurant: It has n cooks. As a guest AKA client you can’t influence this number to get your diner faster. You have to wait. You get in a queue. Same thing as if you take this Magic Bus

      «The Crux of the Biscuit is the Apostrophe»