Installing CPAN Modules within BEGIN of PERL Script

GhostCode has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks,

I am working to develop a software distribution that will be run by several users in my environment. In order this this application to function correctly, I must have the ability to dynamically install CPAN modules when found not to be present within the current PERL installation. I have the logic down; however, I am still having issues...

Logic:
1.) Test if module is currently installed with eval "require <module>";
2.) If module is not installed, install it...

Part two is where I am having problems. I have tried both the command line for cpan (cpan -I <module>) as well as the install function of CPAN::Shell (CPAN::Shell->install(<module>)). Unfortunately every time I run this script it tells me the latest version of the specific module is already installed; however, it most certainly is not. Take the example of Bone::Easy. When I run: CPAN::Shell->install("Bone::Easy"); or `cpan -I Bone::Easy`; I get the message that I already have the latest version. If I then, go on to attempt to use this module (using: require Bone::Easy;), I am met with this module not being installed.

Can anyone help me to better understand what may be going on here. I am certain I am missing something and would really appreciate the help.

Here is a copy of the script I am using to test this procedure...

use CPAN;
use CPAN::Shell;

BEGIN
{
    my @Required_Modules    =    ("LWP::UserAgent","HTTP::Request::Com
+mon","common::sense","Silly::Werder","Bone::Easy");
    my @Installed_Modules    =    ();
    my @Needed_Modules        =    ();
    my @NotInstalled_Modules=    ();
    my @Die_If_Missing        =    ("LWP::UserAgent","HTTP::Request::C
+ommon","common::sense","Silly::Werder","Bone::Easy");
    foreach (@Required_Modules)
    {
        #Eval require * for each module required.
        eval "require ($_)";
        #See from output if module is not already installed - if so, t
+hen add to list of modules needing install.
        if ($@ =~ m/^Can\'t locate/){push(@Needed_Modules,$_)}
        #If modules appears to already be installed, add to list of mo
+dules installed.
        else {push(@Installed_Modules,$_)}
    }
    #For each entry on list of modules needing to be installed:
    foreach (@Needed_Modules)
    {
        #Attempt to install module using cpan
        #my $CPAN_Install    =    `cpan -I $_`;
        #print $CPAN_Install;
        CPAN::Shell->install($_);
        #Parse CPAN output
        my @CPAN_Error    =    split("\n",$CPAN_Install);
        #push module to list of installed modules unless there was a C
+PAN error.
        push(@Installed_Modules,$_) unless $CPAN_Error[$#CPAN_Error] =
+~ m/^>\(error\): Skipping/;
        #push module to list of modules note installed if there was a 
+CPAN error.
        push(@NotInstalled_Modules,$_) if $CPAN_Error[$#CPAN_Error] =~
+ m/^>\(error\): Skipping/;
    }
    foreach(@Installed_Modules){print ("$_\n")}
    foreach(@NotInstalled_Modules){print ("$_\n")}
    <STDIN>;
    foreach(@NotInstalled_Modules)
    {
        my $Missing_Module    =    $_;
        foreach(@Die_If_Missing)
        {
            die("Could not find / instal critical module!") if $Missin
+g_Module eq $_;
        }
    }
    <STDIN>;
}

PickupLine();

sub PickupLine
{
    require Bone::Easy;
    print pickup;
}
[download]

Best,
Bry

Comment on Installing CPAN Modules within BEGIN of PERL Script Download Code

Replies are listed 'Best First'.
Re: Installing CPAN Modules within BEGIN of PERL Script by dasgar (Priest) on Aug 18, 2021 at 22:17 UTC
Based on your first sentence, I would opt to go another route. One thought would be to write your own module(s) that require other modules and provide instructions for your users on how to properly install the needed modules for your code to work. But then you have the challenge of being tech support to help debug installation issues. And both this route and what you have attempted are depending on internet access to be able to download and install needed modules. The approach that I would take would be to use the pp utility from PAR::Packer to bundle the code into a stand alone executable and then distribute the executable. This eliminates the need to provide support to modify/maintain the proper Perl environment on your users' systems. (NOTE: This recommendation is not about making the code run faster or obfuscating the source code. The motivation is to simplify support and maintenance - i.e. maintain a Perl development environment on an OS similar to what end users will use and just provide the stand alone executable.)	[reply]
Re^2: Installing CPAN Modules within BEGIN of PERL Script by GhostCode (Initiate) on Aug 18, 2021 at 23:02 UTC
dasgar, Thank you for the advice. I am already utilizing PAR::Packer for distribution; however, there is still a necessity for the original code to remain as small as possible. The idea is to have this application only install the modules it requires when necessary - thus modules being installed at runtime. Thanks Again, Bry	[reply]
Re: Installing CPAN Modules within BEGIN of PERL Script by swl (Prior) on Aug 18, 2021 at 22:22 UTC
It's worth having a look at the lazy module on CPAN (with the caveat that I have never tried it myself). Quick edit a few seconds after posting: Also have a look at its See Also section for other prior art.	[reply]
Re^2: Installing CPAN Modules within BEGIN of PERL Script by GhostCode (Initiate) on Aug 18, 2021 at 23:05 UTC
swl, Thank you for the advice. I will play around with lazy a little tomorrow. Another thought I have been pondering, because my packaged application will include CPAN, could I not simply include LWP to pull down the modules and CPAN invocations to install from the local tar? I have done this in another scenario where I have automated download, customization, and installation of PAR::Packer. Best, Bry	[reply]
Re: Installing CPAN Modules within BEGIN of PERL Script by bliako (Abbot) on Aug 19, 2021 at 12:30 UTC
I get the message that I already have the latest version. Do you have multiple versions of Perl installed in your system? Then you would better make sure that CPAN::Shell looks at the right places. This will definetely be an issue in your (commented) code: #my $CPAN_Install = `cpan -I $_`; . Which `cpan` are you running through the OS default shell? A `which cpan` will tell in UNIXes. But anyway, that's commented out. So, it remains to debug CPAN::Shell and find out what the INC is. Perhaps adding `CPAN::Shell->o('debug', 'all');` somewhere perhaps before the install loop will let you know what's going on. I tried your script in my system and it tried to install said modules but failed because of lack of admin privileges. bw, bliako	[reply] [d/l] [select]
Re: Installing CPAN Modules within BEGIN of PERL Script by kcott (Archbishop) on Aug 19, 2021 at 11:47 UTC
G'day GhostCode, Welcome to the Monastery. "I am working to develop a software distribution that will be run by several users in my environment." A bit more information about how you're creating this distribution, as well as how knowledgeable your users are with respect to installing Perl modules, would go a long way to providing you with the best help. I create and modify Perl module distributions on a daily basis as part of my $work. I typically use Module::Starter with a heavily modified Module::Starter::PBP as a plugin. In `Makefile.PL` (see ExtUtils::MakeMaker) I specify the dependent modules and required versions under PREREQ_PM (that's a section in the same documentation). Installing `My::Module` will find any missing modules (or those with earlier versions) and attempt to install them. If you're intending to upload `Your::Module` to CPAN, the work is pretty much done for you: installing from CPAN (with the cpan utility, or other similar programs) will — or, at least, should — install all the dependencies. If your distribution is private, you could potentially just put it in some directory and run ``cpan .`` (see cpan for more on that). I know that's pretty general but, in the absence of more specific details, that's about the best I can do. I don't know if you need help with actually creating modules. In case you do, look in the "Reference Manual" section of the "Online Perl Documentation". There's six entries there whose descriptions start with "Perl modules: ": some or all may be useful. — Ken	[reply] [d/l] [select]
Re: Installing CPAN Modules within BEGIN of PERL Script by Anonymous Monk on Aug 19, 2021 at 14:59 UTC
I'm not sure that this approach would work as you anticipate: installation of modules might require advanced privileges. It also seems clumsy to me to try to do this in this way. Instead, I am reminded of the simple `REQUIREMENTS.TXT` strategy that is used with Python. Perhaps you could write a script that scans your Perl source-code looking for `use` statements and extracting the module names. Then, manually sort and de-dupe the list and winnow out the builtins. Now, another simple script could read that file and attempt to install all of the modules therein listed. This is done one time at installation time ... by hand. The essential idea is very simple and it works quite well.	[reply]
Re^2: Installing CPAN Modules within BEGIN of PERL Script by marto (Cardinal) on Aug 19, 2021 at 15:23 UTC
"Then, manually sort and de-dupe the list and winnow out the builtins" This isn't bullet proof. Some distros ship something called perl, without all of the perl core. If the app wasn't going to be published on cpan, hosting it somewhere and providing a cpanfile would provide a dependable route forward.	[reply]