in reply to Memory issue with large array comparison
One thing that occurs to me also is ... “how often do you do this?” How many other tasks do you do that are similar to this? My thought is that, if this is not strictly a one-off task, a database (e.g. an SQLite file) might stand you in very good stead, vis-a-vis a Perl application that is strictly dedicated to the task. It has obviously and plainly been demonstrated how Perl can knock the immediate requirement out of the park ... now, is this one-off, or are there enough other requirements out there that might be resolved by a clever JOIN? Is it (or is it not) beneficial to write Perl programs that can, by loading a database, circumvent the need to write many more almost-the-same but-not-quite Perl programs? It is an angle worth contemplating; I don’t suggest what is the answer.