Where is this huge data structure coming from? (Are you reading it from a file, is it being generated by some continuous process?) Are there other reasons, besides the integrity check you're doing here, for having to keep it all in memory at one time?

It sounds like the integrity check does not require all the data to be memory resident at once. If the data come from any sort of stream, then instead of an array of hashes, you could just do a "while(<>)" loop over each hash, handling just one at a time. (Unless I misunderstand the nature of the task, which is possible.)