in reply to how to handle large hashes
Have you looked into DBM::Deep?
It can work like a hash and supports files up to 4GB and possibly larger depending on your system.