in reply to Similarity searching

You quote specific parameters, leading one to infer that an actual real-world problem is involved. Can you tell us what it is? Shopping cart analytics?

There was a thread not long ago, that might also lend some insight to your problem.

Is it possible to analyze (a representative portion of) the subject data for clustering—perhaps the 8000-dimensional data can be flattened to much fewer? And where multiple clusters exist, one may divide the set.