in reply to Speed searching HTML docs

I probably didn't explain it well enough originally, the bit that checks for "updates" does a file/timestamp check, if a file is deleted, I catch it and remove it from the search hash.

If I had control over the publishing system, I would never have asked this question ;)
The "publishing system" is currently a shared directory on a server that anybody in the group can update.. and I cannot change that process.

Thanks everybody for the help, I think I will probably go with a scheduled process, it's a quick and easy modification to the current setup, so there's a big benifit there.