Maybe your open filehandles get cloned for every thread? You could instead first create your threads and then later on allocate the resources only needed in one thread.