in reply to Problem in creating process

On the basis of the information provided so far (not withstanding some of it is still missing), the single simplest, cheapest and most effective thing you could do to speed up this processing would be to buy two SSDs and add them to your machine. Arrange for each of the sets of files to be written (or copied) to a different SSD and then run two separate processes, one for each set of files.

Done correctly, that would reduce your overall elapsed time to the runtime of the largest of the two datasets. Ie. If they were approximately equal, it would roughly half your processing time.

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
In the absence of evidence, opinion is indistinguishable from prejudice.

In Section Seekers of Perl Wisdom