Let me speculate a little.

Imagine a VM-level 'thread' was not a whole interpeter running on a single kernel thread, but instead was a set of VM registers, a VM 'stack' pointer and a VM program counter. And capable of running on any interpreter in the process.

Now imagine that the VM queried the OS at startup for the number of hyperthreads/cores/CPU (N), and started 1 kernel thread running a separate interpreter in each. And these interpreters all 'listened' to a single, central queue of (pointers to) threads.

When a threadable opcode is reached, if the runtime size of the data to be manipulated is large enough to warrent it, the opcode splits up the dataset into N chunks and queues N copies of itself plus the partition dataset onto the thread queue ('scatter'); prefixed by a 'gather' opcode. Each available processor pulls a copy off, executes on it's partition of the dataset and stores the results.

The first interpreter finished, will pull the 'gather' opcode. It decrements an embedded count and waits upon an embedded semaphore. As each of the interpreters finishes and goes back to get the next piece of work, it does the same until the last finishing interpreter decrements th count to zero, freeing the semaphore and clearing the 'gather' opcode. That frees up all the interpreters to pull the next opcode off the queue and continue. Each waiting interpreter is in a wait state allow the other interpreters to continue, consuming no timeslices, preemptively multitasked by the OS.

The array has been processed, in parallel, by as many threads are available. No user space locking required. No threads needed to be started, nor torn down. Additional opcodes processed by the threaded code pushes are pushed onto the queue (stackwise).

The same VM thread (register set) forms the basis of user space threading for cooperative multi-tasking like in Java, Ruby and many more. The same thread structure is used for both forms of threading.

Just a thought (or five:).