Re: perl regex or module that identifies bots/crawlers

I don't think anyone has done this already. Check CPAN though.

However, if you do create a bot matcher, I suggest you use Regexp::Assemble to turn that list of bots into an efficient regular expression and more importantly, release your work as a module on CPAN :-)

duff

Comment on Re: perl regex or module that identifies bots/crawlers

Replies are listed 'Best First'.
Re^2: perl regex or module that identifies bots/crawlers by argv (Pilgrim) on Mar 20, 2007 at 21:51 UTC
if you do create a bot matcher, I suggest you use Regexp::Assemble to turn that list of bots into an efficient regular expression Yow--this is great... I wish I could vote twice for that posting. Dan Heller http://www.danheller.com	[reply]
Re^2: perl regex or module that identifies bots/crawlers by arc_of_descent (Hermit) on Mar 23, 2007 at 07:02 UTC
Also check out the Regex::PreSuf module which can build a regex from a word list, not necessarily a list of regexes. -- Rohan	[reply]