For the little i know the matter I suppose that beside the thread implementation you put on the field you must rely on the speed of the filesystem and of all underlying OS specific API call.

I doubt that more CPU can fetch a physical hard drive faster than a single one.

My experience fighting with windows is very long, so long to let me say that is by far better and faster to use as much as possible native tools offered (or well, concelead) by OS. An example is fetching file permissions: old Perl modules exhisted but wrapping around tools like icacls.exe is faster and less error prone and works for decades.

So going for native solutions I think the best would be to read the Master File Table directly: this intrigue me a lot but i suspect is a task by far beyond my hackery skills.

Read the MFT is easier than write to it and some tool can do it, so is feasible. Look at ultrasearch and at this discussion that points to swiftsearch on sourceforge

Also the Linux NTFS file system driver can be a reach source of information, if you are able to investigate a Linux driver.

The task of reading MFT can be accomplished in other languages: see a very detailed answer on stackoverflow about C# and this python example

See also analyzeMFT ntfswalk MTF_parser.

For the thread part you can be interested in the marioroy's MCE that comes with many useful examples. At the monastery marioroy shown an exmple of dir walking using MCE in the thread Re: Perl threads - parallel run not working

Update: ATTENTION play your MFT tests on a test harddisk because the risk of corruption is always present!

Good luck and share your improvements!

L*