Well, i have to regularly index a few million documents for a small intranet search engine.

Then you asked the wrong question. The right one is: "What is the fastest way to index a few million documents for a small intranet search engine?"

The answer, as I recently learned from tachyon, is Swish-e. Of course, you'll also want to grab the Perl interface, SWISH, from CPAN.

-sauoq
"My two cents aren't worth a dime.";