in reply to memory:: grep vs looping through hash

To conserve memory, iterate over the hash with each. Invoking keys will create a list, which, for large hashes, could be substantial itself.

Then again, access to the second hash may hit memory in random patterns and thus perform sub-optimally. Large datasets are best managed using a database.

Replies are listed 'Best First'.
Re^2: memory:: grep vs looping through hash
by f77coder (Beadle) on Aug 18, 2014 at 16:47 UTC

    Many thanks for the replies.

    I've considered databases but the incoming data is fairly dirty and cannot be dumped into a dB until it has been cleaned/munged quite a bit, but I've only looked at traditional sql and hadoop.

[reply]

In Section Seekers of Perl Wisdom