Then your hash insert is only taking 5 seconds (all other things being equal).

There is the memory consideration, also (as stated below).

Is this 20M records totalling 1GB or 1 TeraByte? (You mention 1,000 GB in your original post).

Is there a reason you are using a hash? (in your example it looks like you could use an array, but I understand it is merely a "test")

If you have many files (and it sounds like you do) - you could slurp in the entire file (one file at a time) and do the inserts, which will increase your memory usage but decrease CPU time. See File::Slurp