in reply to memory:: grep vs looping through hash

The admonition to “try it” really is the best advice that can be given.   If your system has its memory-back up against a memory-wall, and if in your case grep is going to return long lists such that your system simply has nowhere to put them ... then you probably can’t use it.   You have to use a loop.   However, in that case, “you have bigger fish to fry,” because you are seriously over-loading your system.

There’s a reason why early systems used the term “SOS (as in ‘... --- ...’ ) = Short On Storage.”   If you seriously over-commit the RAM resource, then the system is going to start thrashing, and it will be the futile disk-I/O that sinks the boat, not any clever algorithm or the absence thereof.   If you literally have stuffed your machine full of all the RAM that a 64-bit motherboard can possibly take, then you simply have to expand the job to multiple machines, each similarly equipped.

And you are probably equally screwed whether you use a loop or not, because it is memory references that cause page-faults.   When memory-references that ought to take nanoseconds consistently start taking milliseconds ... “ding dong, you’re dead.”