My method avoids that flocking problem...

There was no flocking problem until you created it.

Its simple:

Simple? Take 1 big input file and split it into N bits.

Run N thread reading those N bits (concurrently, causing the disk head to dance all over the disk), and then write all the records read down N pipes.

Then have a select loop that brings all the records from all the pipes back together in a single thread before deciding what to do with them.

You really don't see that all you've achieved with those 10 threads and 10 pipes is the equivalent of:

while( <HUGEFILE> ) {
    ...
}
[download]

Except it will run at least 10 times more slowly, regardless of how many cores your processor has!

That really isn't just glaringly obvious to you?

This is simple. And fast. And platform independent.

No threads. No pipes. No pointless select loop (which doesn't work with pipes on some platforms anyway)...