First, some suggestions unrelated to the benchmark issue:

Use strict (in code over 1 line long, it will save you countless hours finding little bugs-- it will also enforce some good scope parameters more carefully, it looks like you're having some "fun" with that piece, too. That is, you shouldn't have to undef a list that is about to go out of scope.)

Use CGI.pm (it may be a little heavy, but it provides good form handling)

Use taint mode (you never know what's going to sneak in on a POST).

Your loops will be more readable if you put an obvious list into the definition-- rather than using the C-like syntax. I mean, we can all figure it out, but that takes brain time on the reader end, where it is least desirable.

Have you tested your split against strings containing more than one \W character in a row? Two spaces in between keywords is going to slow your search down for no apparent reason (and may even cause other problems).

Finally, with respect to your algorithm, you can't get away from the performance hit the way you have this built. sysseek may be very efficient, but doing it twice will take twice as long (depending on where in the file the words fall). It looks like you're already using your filesystem to the best advantage, by sorting/indexing the search files by initial letter groupings, etc. but again, if you do something twice it takes twice as long.

In reply to (ichimunki) Re: All-Perl search engine having speed issues by ichimunki
in thread All-Perl search engine having speed issues by Anonymous Monk

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.