Ya, I've been thinking about groupings. Given the patterns

datum_1 = x and datum_2 = y and datum_3 = z
datum_1 = x and datum_2 = a and datum_3 = d
datum_1 = x and datum_20 = g and datum_13 = j
[download]

first find alll lines for which datum_1 = x, the just work on those for ll other patterns that begin with datum_1 = x. Doing so, I figure I can reduce the number of line sans from 40-billion to ~400-million given the average number of repeat datums in patterns. A nice reduction, but still interested in better. I'll probably end up just putting it all in a database and let the machine do the heavy lifting...