Because what you are now attempting to do "in memory" exceeds your actual memory capacity by a factor of at least six, you are now actually using a
which has utterly no idea what you are doing. And you are right now beating it to death. One short-term possibility might be to make use of the "tie" facility, which at least would allow you to "tie" your hashes to something which could more-reasonably implement the same algorithm that you are doing right now. Start with