zentara has asked for the wisdom of the Perl Monks concerning the following question:

Hi, my snippets collection is getting kind of large, and my old method of letting Perl grep through all the files is getting slow and "hard drive intensive". So I have a preliminary design using swish++ and then dumping the files returned from swish into a Tk listbox, and then you can browse the file with the search word highlighted.

So I figure this has been done before, and was wondering if there are better ways. (Before I get too deep into it :-)

The only drawbacks I see is swish++ likes to ignore alot of short common words, which are Perl keywords...like for, if ,etc; and it is limited to single word searches.

So has anyone else found a method which can limit the search to 1 file, like swish++ ?

My first inclination was just to "cat" all the files together in a big text file, and just search it. But then what would be the best way to compress and search it, retaining the original filenames?

My intuition tells me I want to convert the text to some compressed binary reperesentation, to make the searching faster, and the file smaller.

So I would appreciate any ideas and pointers which anyone may have.(before I go sailing off in the wrong direction :-) ) TIA

I'm not really a human, but I play one on earth. flash japh
  • Comment on looking for ideas for managing snippets collection

Replies are listed 'Best First'.
Re: looking for ideas for managing snippets collection
by cyocum (Curate) on Apr 29, 2004 at 09:28 UTC

    I am not familar with swish++ but there is a perl solution that might take some coding but will give you good results in the end. It is called Plucene and is a Perl port of the Java Lucene search program. I hope this helps.

[reply]
Re: looking for ideas for managing snippets collection
by Anomynous Monk (Scribe) on Apr 28, 2004 at 23:54 UTC
    "Letting Perl grep through the files"? How about letting grep grep through the files? It's hard to imagine you could have that much in the way of snippets.
[reply]
      It's hard to imagine you could have that much in the way of snippets.

      Well over the last few years, I've accumulated almost 256 Megs of snippets, docs, and html related to Perl. I used to be a "compulsive garage shopping junkie", but now I invest in collecting Perl. :-)

      I'm not really a human, but I play one on earth. flash japh
[reply]
Re: looking for ideas for managing snippets collection
by artist (Parson) on Apr 29, 2004 at 14:19 UTC
    As a learning approach, you can build the index, as you go and use the scripts. While working with various files, I use 'etags' from emacs.
[reply]

Back to Seekers of Perl Wisdom