Re: Help on building a search engine (design / algorithms / parsing / tokenising / database design)

in reply to Help on building a search engine (design / algorithms / parsing / tokenising / database design)

This node matching comments by Eugene has some pointers to comment extraction - be warned it's not as easy as it seems to get all the special weird cases, especially where there are multi-line comments or worse nesting coments. I would make the comment extractor conservative and cautious so you don't end up with a great tract of source code in your db by mistake.
Andy

Comment on Re: Help on building a search engine (design / algorithms / parsing / tokenising / database design)

Replies are listed 'Best First'.
Re: Help on building a search engine (design / algorithms / parsing / tokenising / database design) by bobtfish (Scribe) on Jun 07, 2004 at 12:37 UTC
Just what I was looking for, thanks :) Now I've just got the how to make it lightning fast question to resolve.	[reply]

In Section Seekers of Perl Wisdom