Couple thoughts I had:
Run it through Devel::DProf.
But as a guess, my bet it would show the DB access would be your biggest lag. I would suggest using a bigger DB like PostgreSQL or MySQL then profile your indices.

I haven't used MySQL much in the last couple of years, but PostgreSQL offers the excellent explain tool.