If I understand this correctly, then he intends to send the same data to every thread. Thus it is not really the traditional Boss/Worker pattern as every worker will perform the exact same (not similar) task.
I have had a similar requirement with realtime data and I did not manage to get this working as unfortunately there is no way to have a shared array of Queues.
Only way would be to store the read data in memory before starting with the threads, but this means a lot of memory consumption, which will be high already with 100 threads.
Update: I would be interested i there is a decent way to achieve a shared array of Queues. | [reply] |
No, the data sent to each thread at the beginning of a run is different, and shifted off of an array or something. The key is for efficiency...STOP multiple spawning and/or forks...and reuse the threads, just reset them and refill with fresh data to be processed.
We may be misunderstanding the fine points of what he is attempting, but reusing threads is the best for efficiency.
| [reply] |