I'm comparing two huge files with unique structure, for this matter I'm reading each file into a hash structure, and compering the two hash.

If you're approaching the problem this way, “you're already dead, and using multiple threads won't really help you.” You're moving a huge file into another huge file (the virtual-memory store), and comparing them in a way that's certain to produce page faults. Your program is doomed to run like a constipated snail...

Take the two files, sort them using a disk-based sort, then compare the two sorted streams. The sort operation will be very fast and efficient, and the file comparison will proceed like greased lightning. You don't have to “search for” anything.

Welcome to COBOL ... to sort and to merge. The techniques born of necessity eighty years ago (before commercial computers were invented...) still work. Only better.

Your program will work literally hundreds of times faster than before... even on a single CPU.

“Don’t diddle your code to make it faster:   find a better algorithm.”
– The Elements of Programming Style (Kernighan)