You have to put one of the two files into a hash, it doesn't really matter which one. Since the hash won't fit into memory, it must be put on disc, either into a real database like mysql or in this case better into a DBM::Deep database.
Then just loop through the other file and look up the ids in the hash. Should be a speedup million fold speedup, literally.