did you try a database solution?

I did. It was so spectactularly much slower that I didn't bother posting it (and I used postgres).