Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hi!

Does anyone know any good tutorials/papers on search engine design? e.g. if I want to use a RDBMS, what is the best way to organize my tables? How would I organize my data/tables if I want users to be able to search by category? That kind of stuff...
Any personal recommendations based on experience are greately appreciated as well.
I know Perl pretty well (I think:), so coding is not a problem. I just want to avoid common mistakes by reading as much as possible on the subject.
I tried looking on the web and here (using Search) but could not find anything appropriate.

Thanks in advance!

Replies are listed 'Best First'.
(Ovid) Re: Search engine design
by Ovid (Cardinal) on Dec 29, 2001 at 01:42 UTC

    Check out Perlfect Search. It's easy to use, fairly customizable and will save you a lot of development time. I had my first search engine up and running in just a few minutes with it. It's also available under the GNU license.

    Perlfect isn't perfect, but it does allow you to either generate a search through static text files or spider a site to generate a search database for a dynamic site. It's very handy that way.

    Cheers,
    Ovid

    Join the Perlmonks Setiathome Group or just click on the the link and check out our stats.

[reply]
Re: Search engine design
by grep (Monsignor) on Dec 29, 2001 at 02:11 UTC
    There are 2 excellent articles in Linux Journal from thier "At the Forge" columns.

    At the Forge: A Simple Search Engine
    At the Forge: More About Searching

    Reuven's columns are aimed at novice CGI coders, they are almost always in Perl.

    He discusses some simple concepts about search engines, but I would have to agree with Ovid. If you find something that works, don't reinvent the wheel. If Perlfect does not work you these articles should be a good starting place.

    grep 
    grep> cd pub
grep> more beer

    [download]
[reply]
[d/l]
Re: Search engine design
by jryan (Vicar) on Dec 29, 2001 at 02:44 UTC
    Although grep and Ovid both gave excellent places to start, please allow me to suggest 2 more if the others don't help enough. First of all, take a look at the NMS Simple Search for an example of a very basic search engine. If that isn't enough power for you, you can try the NMS Advanced Search, but be warned that it is still considered pre-alpha (although it seems to be running fairly stable thus far). You can try the advanced search out here.
[reply]

Back to Seekers of Perl Wisdom