I was writing my very own search engine (TM) and had the index in a mysql database. To process the queries I had to intersect lists with up to several million items. I found that, whatever I tried, mysql was too slow. My solution was to move the core of the intersection part of the engine to a custom C program with its own binary file for the index.

This may not be pretty, but it was the only way I could find to get fast results and use a reasonable amount of memory (the overhead of Perl arrays becomes problematic when you have millions of items!).

So I think that in some cases it is OK to move some operations out of the database if there is a clear benefit. I was getting a 100x speedup or better in some cases, but I don't think I would do it for just a 20% speedup. YMMV.