When you say "functionality similar to that of google.com" what do you mean exactly? When I hear that I'm guessing you mean that you need something of similar search result quality and depth of indexing. If that is the case stop right now and go buy the darn thing from Google. You won't get there on your own.

As a reference point, I once built a search engine combining Apache/mod_perl, MySQL and Glimpse. It took around 4 months to complete working alone. It indexed all of the Open Directory project and served most queries in under a second running on a PII/600. The search result format was actually more complicated than Googles - it included the category hierarchy and had advanced tree-limiting features.

The project was generally successful. However, it never came close to providing something comparable to Google. Why not? The search results sucked, to put it mildly. All it did was a simple partial-word match. Glimpse supported more but the more advanced features were too slow to use. Also, the indexing was really really slow. It would never scale to indexing the entire Internet no matter how much hardware you put behind it. As it was it took around 6 hours to index the Open Directory database (although much of that was in character-set translation).

So, in short, be very careful about what you attempt here. If you need Google, buy Google (or one of the competitors like Verity, etc.). If you can make do with much less then you might build it yourself. But have no illusions about what you'll end up with.

-sam

In reply to Re: Perl Search Applicance by samtregar
in thread Perl Search Applicance by PyroX

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.