Have you actually tried tieing your hashes to a DBM file? You may find the speed hit is not anywhere near as bad as you think.

I have DBM files with many millions of keys, and fetches are still lightning fast.

Considering it's probably about 4 lines worth of code to try, give it a shot :-)

I don't think it's even worth trying to draw conclusions about how fast a DBM file access will be by looking at the slowdown of a swapping process. Apples and Oranges.