In Windows, there is a mechanism like to muse about to watch the system load. You can process things from a queue such that there are n active threads working on it; if one becomes blocked, it will start something else, keeping the CPU busy.

—John