What modules are we actually using?

cbrandtbuffalo has asked for the wisdom of the Perl Monks concerning the following question:

Is there a standard module or regex out there to search a file system and list all modules that are actually being used?

We're preparing for a perl upgrade, and I want to avoid having to re-install modules that we no longer use. For example, we currently use DCE for authentication/authorization, so we have the DCE module. But we are moving off DCE, so we'll never need that module again.

I started writing a simple script to look for 'use', 'require', 'do', etc. but there are enough permutations that it started to seem non-trivial. There are other tricky bits like DBI and the RootClass option that is like a use, or Class::DBI with the inflate option that can indicate a module to use.

Anyone do this already?

Update: I'm starting to focus my problem based on the comments thus far.

I want to try to do this without running the actual code. Specifically, I'm going to run some script against our Perforce depot (version control repository) because I feel I can trust that if it's in a prod mode there, it's running somewhere in our environment. Plus I can run the script any time without involving a bunch of other support people.
By default, this limits what I can look at. As mentioned in Re: How do I detect what modules are not being used? and other places, you can't detect modules when the module or method names are generated dynamically. I don't think we do this much.
I'm only looking for CPAN modules used by programs, not necessarily in-house modules. The difference is our in-house modules are all supported and the support person should know if it's being used or not.

So I'm considering going ahead with what I started which is to work file-by-file and look for all the ways I know of to pull in a module (use, require, etc. see above). I'll create a test file that uses all of them and add test cases as they come up. I think I'll start with Module::Info as suggested below and just add cases to that.

Comment on What modules are we actually using?

Replies are listed 'Best First'.
Re: What modules are we actually using? by davidrw (Prior) on Apr 27, 2005 at 14:10 UTC
Have a look at this thread: Which modules does somescript.pl use? It covers solutions involving looking at %INC or using Devel::Cover	[reply]
Re: What modules are we actually using? by eXile (Priest) on Apr 27, 2005 at 14:19 UTC
just a thought: You could walk all your modules and see if the access time of the module files are something recent. Combined with http://perldoc.perl.org/perlfaq3.html#How-do-I-find-which-modules-are-installed-on-my-system- that should give you a possible solution.	[reply]
Re^2: What modules are we actually using? by Booger (Pilgrim) on Apr 27, 2005 at 14:47 UTC
Having tried this sort of thing before, I think that your suggestion is a good one. Figuring out which modules are being used is an extremely difficult task (given that some of your modules may be included at run-time) and even the best solutions for walking a codebase and discovering module usage don't quite cut it.	[reply]
Re^3: What modules are we actually using? by dmorelli (Scribe) on Apr 27, 2005 at 15:02 UTC
I was thinking about this too. It sounds like a tall order. Maybe it's unlikely to be happening in the OP's particular code base, but it's entirely possible to require an arbitrarily-named file that itself is requiring modules and doing things with them. And eval could be used in a similar manner. Hard to say where code may be coming from.	[reply]
Re: What modules are we actually using? by tlm (Prior) on Apr 27, 2005 at 15:00 UTC
You may find it useful to read the replies to How do I detect what modules are not being used?. the lowliest monk	[reply]
Re: What modules are we actually using? by eric256 (Parson) on Apr 27, 2005 at 16:01 UTC
Just an idea: Could you insert a coderef into @INC that would track which modules where loaded and record that info into a file or database. Then you run your shop as normal for a period of time and know what modules have been used. If you modules always use Strict maybe you could bastardize it to modify the @INC for you. I'm not sure that is the perfect solution, but it might be a good way to see what modules actualy get used, it would require all the scripts to be run at least once though to ensure that there dependencies where captured and some things will have dynamic dependencies which will be trouble but you probably know what most of those are already. Good luck and let us know if you find some spiffy solution. Update: If you add `BEGIN { # look in cwd first unshift @INC => sub { warn "Loading $_[1]\n"; return undef; }; }` [download] to the top of strict (still not sure this is the best place to put it, but it works as a kludge) then it prints all module uses to STDERR. You would of course want to modify it to do something more usefull in your case. ___________ Eric Hodges	[reply] [d/l]
Re: What modules are we actually using? by ghenry (Vicar) on Apr 27, 2005 at 14:23 UTC
Hi, Some thoughts of the top of my head: Module::Info and Module::Find. Maybe you could use these in some existing programs to find out what modules are loaded etc. Maybe you could look at the access time of all the modules in a certain directory? I think these might not work, but worth a thought. Walking the road to enlightenment... I found a penguin and a camel on the way..... Fancy a yourname@perl.me.uk? Just ask!!!	[reply]
Re: What modules are we actually using? by Dr. Mu (Hermit) on Apr 29, 2005 at 05:19 UTC
This may be a little off-the-wall, but it worked for me. I wrote a program for distribution under Windows and needed a complete module list so I could check their licenses for possible restrictions. I was using perl2exe to package the program and had successfully packaged an installation that would run on a perlless system, a critical prerequisite. Then, by running perl2exe in verbose mode, I got a complete list of all the modules included in the build. (I was pretty astounded by how many there were!)	[reply]
Re: What modules are we actually using? by Tanktalus (Canon) on Apr 27, 2005 at 21:06 UTC
Actually just looking for the use, require, and do (??) calls may be sufficient. (You use "do" to load CPAN modules?) Personally, all of the CPAN code I use is used either via 'use Bare::Word' or 'require Bare::Word'. If I were to grep through all of my code for that, uniq that list, and then remove the in-house modules, that would be something I could feed to CPAN. Even if those modules were to require other modules, the CPAN module would handle that for me. Sometimes, a simple solution with minor manual steps is sufficient ;-)	[reply]
Re^2: What modules are we actually using? by cbrandtbuffalo (Deacon) on Apr 28, 2005 at 19:00 UTC
That's basically where I landed too, and the approach I'm going to take. I started looking at Module::Info and it already does the 'use' and 'require' parts. I'm going to try to add: option to exclude pragmata; it currently spits out stuff like 'strict' from 'use strict' as a module search for RootClass, which can be used in DBI connect search for Class::DBIs has_a which can inflate something into an object of a specified class; I know we have the last two items in our code base for sure. As others turn up, I'll add them too.	[reply]
Re: What modules are we actually using? by clscott (Friar) on Apr 28, 2005 at 14:35 UTC
No one has metioned it yet so I would like to point out Module::Dependency -> http://search.cpan.org/dist/Module-Dependency/. From the documentation: This is a suite of modules (and a set of helper programs that you may use if you wish) that allow you to investigate the dependencies between perl programs and modules. This is primarily aimed at the programmer/ developer audience. -- Clayton	[reply]


Keep It Simple, Stupid
	PerlMonks