Re^2: Threading and join/termination question

That’s a very-terse example of a million-dollar idea. (Great example if you’re a serious Perl-head ... not-so-easy if you’re not.)

The idea is this: instead of defining “a thread” as corresponding to “a unit of work to be completed,” and then trying to “limit the number of” those threads, create a pool of threads, calling each one of them “workers.” Then, give those workers a production-line queue of things-to-do. Each worker grabs a request off of the production-line, does it, shoves the results downstream somewhere, and then grabs another one ad infinitum, surviving until the production-line finally runs dry. The number-of-workers (thread-pool size) should correspond to the maximum number of such units-of-work that you know this piece of hardware can predictably accomplish. So, whether the production-line queue might be short or long, you remain confident that the work is being carried-out as fast as possible ... no matter how short or long the queue might be, the “completed units-of-work per second” will remain steady. If you are gifted with faster hardware, you simply turn-up the number-of-workers knob a little bit more. (And if you are gifted with a cluster of units, spread the work out among all the CPUs you have, so-many workers apiece.)

Replies are listed 'Best First'.
Re^3: Threading and join/termination question by SimonPratt (Friar) on Aug 01, 2014 at 10:27 UTC
I really have to agree with sundialsvc4. Threads are so expensive to create that you are almost aways much better off to create a set number of threads (usually no more than the physical number of cores you have, for optimal results), then using a queuing system, such as Thread::Queue, to distribute work until all the work is complete. Also, unless you are creating a thread that is performing a never ending task (such as listening to events from an external source), its generally not a great idea to detach them, especially if you are interested in when they complete or what they return.	[reply]
Re^3: Threading and join/termination question by cormanaz (Deacon) on Aug 01, 2014 at 12:54 UTC
Actually I was wondering about that...are there rules of thumb for how many threads a given platform can deal with? Equal to the number of cores? Limited by memory?	[reply]
Re^4: Threading and join/termination question by SuicideJunkie (Vicar) on Aug 01, 2014 at 15:16 UTC
As a first pass estimate, consider your chokepoints. Is the program mostly doing disk IO, number crunching, using a lot of memory, network bandwidth, special devices, or just idle? If disk I/O is the bottleneck, then adding a second thread would be likely to cause the disk to thrash and slow down. If math is the bottleneck, then the number of cores in your box is likely to be the sweet spot. Something for each core to do, without making it context switch too often. Memory limitations are pretty easy. If your task takes up a lot of RAM, limit your threads to a number that won't use up all the RAM. Once your threads start having to swap to disk, performance will drop through the floor. If network traffic is filling up, then choose a number that is enough threads to get a good % utilization without saturating the link. (Also keep in mind that the local bandwidth on your network card is very different from the bandwidth you'll get through your final connection to the internet. Percentages will vary, and be sure to divide by 8 when you see speeds measured in megabits per second) If you've got a bank of special devices to operate, then you may want one for each device to keep them busy without being wasteful. Depending on the task, one thread may be enough if it can deal with each device occasionally in a loop, such as a bank of printers. If you are idly waiting for something, then a single other thread is handy to do a blocking wait while your main thread putters around keeping a GUI responsive.	[reply]