in reply to wait till memory is flushed
Another possibility would be to use something like BerkeleyDB (or GDBM_File or whatnot) and leave the hash on disk instead of in memory.
The cake is a lie. The cake is a lie. The cake is a lie.