note
Ryszard
After looking at various algorithms and what not, I've settled on a (sigh) ready made solution, [http://www.swish-e.org/|swish-e].
<p>Its pretty cool, and while written in c, has a perl API which is quite nice. Its extremely fast for filesystem index scans and terribly slow for web spidering (not surprising really), (props [tachyon]).</p>
<p>My site is a dynamic photo album site, not static html pages. As the content I want to index is stored on the FS, the scan takes less than a second (as opposed to 33hrs spidering) with swish-e.</p>
<p>I've learned (not heaps) but a little more about search engine's, and know a bit more about the different algorithms and the advantages/disadvantages of them.</p>
<p>Again, unfort, the Vector Space alorithm referenced above did not meet my needs, with completely irrelevant results, YMMV. I've not delved into the nitty gritty of how/why, but its on my list of things to do.</p>
353299
353299