Okay, So, in the vein of what BrowserUK is suggesting, and based upon your better description of the problem, what I would look to do is:
- Fire up a fixed number of query threads.
- Associate one output Queue with those -- take responses from each of the query threads and pre-pend it with data about which client it came from, etc (used later on)
- Fire up a fixed number of comparison threads
- Associate an input Queue with each of these
- Use the main program as the controller -- to pull data from the query queue, and use the pre-pended data in order direct the flow to the appropriate comparison queue
- Optionally associate an output queue with all workers so that they can send their final status (comparison good, bad) back to the main polling loop. note if you do this, you will likely want to use the same sort of magic with pre-pending some control data into the queue data.
- This will allow you to do flow control on the queries (so that you don't get too backed up on the comparison workers), as well as to tightly manage when new queries get fired off (no sense in firing off too many outstanding queries if you don't have any comparison workers available). It will also allow you to scale slowly and surely.
You could get a 1:1 (query to comparison) model going very quickly, then add a couple or few queries to test all of the flow control, and then scale in some more comparison workers.
Just my $0.02