If you find that searching gets to be a performance bottleneck, one thing you can do is to build custom indices for the pages in the database. You can build an index for each word in the text, and another with word pairs in the text. (This, for example, would have an entry "if you", "you find", "find that" and so on) Searching for phrases is just a matter of splitting the phrase into pairs and searching for documents that match all the pairs. (It's generally good enough)

You might not have to do this--there's only 180K pages here, so full-text searches may very well not be performance bottlenecks at the moment.

In reply to Re: Re: Re: Offsite Perlmonks Search Engine by Elian
in thread Offsite Perlmonks Search Engine by blakem

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.