Can you use some disk space? Personally I would tie a hash to a BTREE on disk with DB_File, scan one file into the BTREE, then go through the other and do my test. The buffering is not essential to the performance, but a BTREE will happen to buffer that quite nicely to perform very well.