If you could arrange the 35,000 patterns into a sequence of tests in the right order, you could save a lot of work. Say that pattern 1 requires a value of 1 in column 3, and pattern 2 requires a 2 in column 3. Suppose you tested that first. If the value was 1, then pattern 2 is eliminated. If the value is 2, pattern 1 is eliminated. If it is neither 1 or 2, both are eliminated. So you should look for columns that are tested in many patterns. Such columns are also good candidates for indexes (using
BerkeleyDB, for example).
A sequence of tests like this is called a decision tree. Creating a perfectly optimal one is an NP-complete problem, but there are methods for making fairly good ones. You might check out the module AI::DecisionTree for ideas.