The OP wants to go to a situation where different computers are used. That's never going to work with threads.

Look again! The OP want's to service multiple concurrent processes, by feeding them unique subsets of a large common dataset. Those external processes cannot make use of COW shared data, it has to be fed to them via a file or pipe. All that is required to extend the threaded solution offered to use multiple machines, is to substitute rsh or ssh for open2.

The OPs program starts one thread to perform the subsetting and feeding of that subset to an R instance. No messy interleaving of reading and writing to multiple sources and targets. Each thread performs the subsetting; starts the R instance; writes the data; reads the results; shuts down the process (or feeds it another). Whether that R instance runs locally or remotely is entirely transparent to the threading