in reply to Merge two huge datasets by ID

If the files really are huge, I'd probably use the unix sort command line tool to sort the file by id before attempting to merge them.