How about reading the tables into a database and using SQL instead? ... Or, even easier but slower, ...

Yes, you could use a database. This might even be the best solution if the files are truly huge and can't fit into memory.

But if speed matters (and assuming the data is not too large for the available memory), the hash solution I suggested would be completed long before the data is stored into the database and you even start to query the database.