I've just read an article about Swish-E, this one looks like a little nice indexer and searcher (especially being able to index and search man pages is a great feature) but I'm not sure if it can handle thousands of files which sum up to more than hundreds of MB of data (also deleting, modifying files and reindexing performance issues, etc.)