If you found the module was too slow, you're clearly talking about huge sequence, not merely a few thousand characters. If you're trying every pairing, seems to me that's going to be N * (N-1) / 2 pairs. You can choose any one of N for the first, but one fewer for the second; A-B is the same as B-A.

All pairings means you need random access, so you want them in memory. Huge means You're going to use up your memory. Have you considered using a simple memory/file database, maybe BerkelyDB or SQLLite, to help manage the storage and swapping?

is there any way you can do some hierarchival competition, rather than all-pairs?