Searching for Modules and Descriptions on CPAN Remotely

ghenry has asked for the wisdom of the Perl Monks concerning the following question:

Dear Master Monks,

My goal is to be able to search for Modules on CPAN with the same prefix, grab the complete name and description, then print/save the results.

I have been playing with Parse-CPAN-Modlist and CPAN-Search-Lite, specifically CPAN::Search::Lite::Info.

Initially, I thought I was on to a winner, i.e. grab the necessary files to search from (03modlist.data.gz etc.), save them to a tempdir, parse them and print/save/process the results. But both modules insist on only processing Released modules, and I want to process/search every module.

I could just use WWW-Mechanize, and search by hand, but I didn't want to keep hitting http://seach.cpan.org every time.

I also don't want to write all of it myself, but looks like I might have to, unless you guys have any other suggestions of modules/methods?

Thanks for your time,
Gavin.

Walking the road to enlightenment... I found a penguin and a camel on the way.....
Fancy a yourname@perl.me.uk? Just ask!!!

Comment on Searching for Modules and Descriptions on CPAN Remotely

Replies are listed 'Best First'.
Re: Searching for Modules and Descriptions on CPAN Remotely by adrianh (Chancellor) on Jan 23, 2006 at 12:37 UTC
My goal is to be able to search for Modules on CPAN with the same prefix, grab the complete name and description, then print/save the results. Might be easier to use CPAN or CPANPLUS. Take a look at CPANPLUS::Backend. For example: `use CPANPLUS::Backend; my $cb = CPANPLUS::Backend->new; my @modules = $cb->search( type => 'module', allow => [ qr/^Test::/ ] +); print $_->name, "\n" foreach @modules` [download] To get the module names starting with Test::	[reply] [d/l]
Re^2: Searching for Modules and Descriptions on CPAN Remotely by ghenry (Vicar) on Jan 23, 2006 at 13:46 UTC
How annoying!!!! Just when I thought I'd nailed it, CPANPLUS::Backend is just like the others I mentioned, it only returns descriptions of registered modules! Walking the road to enlightenment... I found a penguin and a camel on the way..... Fancy a yourname@perl.me.uk? Just ask!!!	[reply]
Re^3: Searching for Modules and Descriptions on CPAN Remotely by adrianh (Chancellor) on Jan 23, 2006 at 13:56 UTC
Just when I thought I'd nailed it, CPANPLUS::Backend is just like the others I mentioned, it only returns descriptions of registered modules! Can you define (and give an example module) of a non-registered module that you expect to appear? For example my Test::Class isn't registered and it appears in the list.	[reply]
Re^4: Searching for Modules and Descriptions on CPAN Remotely by ghenry (Vicar) on Jan 23, 2006 at 14:12 UTC
Re^5: Searching for Modules and Descriptions on CPAN Remotely by adrianh (Chancellor) on Jan 23, 2006 at 15:56 UTC
Some notes below your chosen depth have not been shown here
Re^2: Searching for Modules and Descriptions on CPAN Remotely by ghenry (Vicar) on Jan 23, 2006 at 12:56 UTC
Perfect!! Looks like this will do the job and then I can use CPANPLUS::Module to get the description. Walking the road to enlightenment... I found a penguin and a camel on the way..... Fancy a yourname@perl.me.uk? Just ask!!!	[reply]
Re: Searching for Modules and Descriptions on CPAN Remotely by ysth (Canon) on Jan 23, 2006 at 12:54 UTC
There are more modules in 02packages.details.txt.gz than in 03modlist.data.gz, but still only the latest non-development release of each module is listed. Perhaps what you want is to go through authors/01mailrc.txt.gz and pull the CHECKSUMS file for each author?	[reply]