Upgrading XS module to later version

clerew has asked for the wisdom of the Perl Monks concerning the following question:

First my setup:

Ubuntu 14.04, with Perl installed in /usr (and one doesn't tinker with /usr)
/usr/local/CPAN/ for builds, etc
/usr/local/lib/perl/5.18.2 for downloaded modules; this is earlier in @INC than official /usr stuff, so stuff here should supersede stuff there.

Now the problem

List::Util (as supplied in /usr) is version 1.27
I want to download DateTime, which Requires version higher than 1.27.
Methinks "I will install later version (1.42) in /usr/local/..., where it will be seen before the older one".
But install List::Util says "but it's already installed"
Making the version in /usr invisible breaks cpan.pm, which itself makes use of List::Util
Eventually, after test List::Util followed by manual Makefile install, I land up with the following:

In /usr/local/lib/perl/5.18.2:

    i686-linux-gnu-thread-multi-64int
    List
      Util.pm
      Util
        XS.pm
[download]

where Util.pm is the full compiled binary

But, since /usr/local/lib/perl/5.18.2 does not contain any List directory, installing DateTime fails because it still sees the old List::Util

So I grabbed /usr/lib/perl/5.18/List/Util.pm (the one I originally tried to make "invisible") and hacked it to look like

our @ISA        = qw(Exporter);
our @EXPORT_OK  = qw(first min max minstr maxstr reduce sum sum0 shuff
+le);
our $VERSION    = "1.42";
our $XS_VERSION = $VERSION;
$VERSION    = eval $VERSION;

require XSLoader;
XSLoader::load('List::Util', $XS_VERSION);
[download]

and inserted it into /usr/local/lib/perl/5.18.2 giving

  List
    Util.pm ---(as hacked)
    Util
      XS.pm ---(copied from the i686-linux... one)
  i686-linux-gnu-thread-multi-64int
    List
      Util.pm ---(the new binary)
      Util
        XS.pm
[download]

But this still picked up the old 1.27 binary, and I got the ridiculous situation

cpan[1]> m List::Util
CPAN: Storable loaded ok (v2.41)
Reading '/usr/local/CPAN/Metadata'
  Database was generated on Tue, 16 Jun 2015 21:53:20 GMT
Module id = List::Util
    CPAN_USERID  PEVANS (Paul Evans leonerd@leonerd.org.uk>)
    CPAN_VERSION 1.42
    CPAN_FILE    P/PE/PEVANS/Scalar-List-Utils-1.42.tar.gz
    UPLOAD_DATE  2015-04-23
    INST_FILE    /usr/lib/perl/5.18/List/Util.pm
    INST_VERSION 1.27
[download]

Incidentally, I tried to revert to the old situation by renaming the new List as Lost, but it still gives the same result to m List::Util so there must be some secret cache somewhere which remembers that state, even after a logout/reboot).
And further incidentally, that /usr/local/CPAN/Metadata file is 28MB long, and contains unreadable garbage; do I really need to keep it around?

So Finally, the Question

What incantations do I have to give to make my List/Util.pm connect to the newly downloaded binary XS version?

And thank you for your patience in reading this far.

Comment on Upgrading XS module to later version Select or Download Code

Replies are listed 'Best First'.
Re: Upgrading XS module to later version by Anonymous Monk on Jun 18, 2015 at 22:52 UTC
I don't have a direct answer handy, but just some experience: If you want to work with Ubuntu's Perl build, then I find the easiest is to work within Ubuntu's system: `sudo apt-get install libdatetime-perl`. Mixing the system Perl with `cpan`-built modules can sometimes work well, sometimes not, and it seems you've hit one of the latter cases. Since you seem to be comfortable with this kind of thing, why not just install a self-built Perl in e.g. `/opt/perl5.18`? That's worked great for me so far; I can install modules via `cpan`, `cpanm`, and manual `Makefile.PL` installs without problems and without worrying about affecting Ubuntu's Perl at all.	[reply] [d/l] [select]
Re^2: Upgrading XS module to later version by clerew (Novice) on Jun 19, 2015 at 15:50 UTC
Yes, using `sudo apt-get install libdatetime-perl` seems a good idea, especially for a basic module such as DateTime, which seems a useful thing to have around. Not so sure I want to clutter /usr with every Cpan Module I fancy trying out and might never use again. So I did that, and then got: `cpan[1]> m DateTime CPAN: Storable loaded ok (v2.41) Reading '/usr/local/CPAN/Metadata' Database was generated on Tue, 16 Jun 2015 21:53:20 GMT Module id = DateTime CPAN_USERID DROLSKY (Dave Rolsky <autarch@urth.org>) CPAN_VERSION 1.19 CPAN_FILE D/DR/DROLSKY/DateTime-1.19.tar.gz UPLOAD_DATE 2015-05-31 MANPAGE DateTime - A date and time object INST_FILE /usr/lib/perl5/DateTime.pm INST_VERSION 1.06` [download] So it had actually installed v1.06 (presumably the latest Ubuntu had got around to) although it knew that a later v1.19 existed. Fortunately, my application did not care about that version, but promptly declared that what it really wanted was `DateTime/Format/Mail.pm`, which then downloaded via Cpan; my application then ran, and hit a bug which I shall now have to sort out.	[reply] [d/l] [select]
Re^3: Upgrading XS module to later version by Anonymous Monk on Jun 19, 2015 at 16:13 UTC
what it really wanted was DateTime/Format/Mail.pm `sudo apt-get install libdatetime-format-mail-perl` Those module versions will always lag behind the current ones, sometimes significantly, but they should always play nice with Ubuntu's Perl installation (that's the whole point). If you want to install a bunch of custom CPAN modules, I'd recommend a Perl built just for that in `/opt`, where you can experiment and it can easily be clobbered and re-installed in case things do happen to get messed up. Either that, or install all modules locally, e.g. local::lib. A messed up system Perl can be tricky to pick apart and revert to a clean state. Also remember that there are Perl scripts that are part of other Ubuntu packages which rely on a working system Perl.	[reply] [d/l] [select]
Re: Upgrading XS module to later version (uncorrupting system perl) by Anonymous Monk on Jun 18, 2015 at 23:16 UTC
What incantations do I have to give to make my List/Util.pm connect to the newly downloaded binary XS version? Download, https://cpan.metacpan.org/authors/id/P/PE/PEVANS/Scalar-List-Utils-1.42.tar.gz, install it , the end	[reply]
Re^2: Upgrading XS module to later version (uncorrupting system perl) by clerew (Novice) on Jun 19, 2015 at 16:11 UTC
Yes, but `https://cpan.metacpan.org/authors/id/P/PE/PEVANS/Scalar-List-Utils-1.42.tar.gz` is exactly what Cpan had already downloaded for me and failed to install (even with manual assistance using the Makefile Directly)	[reply] [d/l]
Re^3: Upgrading XS module to later version (uncorrupting system perl) by Anonymous Monk on Jun 19, 2015 at 20:29 UTC
Yes, but https://cpan.metacpan.org/authors/id/P/PE/PEVANS/Scalar-List-Utils-1.42.tar.gz is exactly what Cpan had already downloaded for me and failed to install (even with manual assistance using the Makefile Directly) Yeah and, how is this relevant? Your complaint is that you've manually edited stuff and now cpan won't do something you want ... great, do it yourself, cpan is just a helper , if its not helping you, do the job yourself	[reply]
Re: Upgrading XS module to later version by choroba (Cardinal) on Jun 19, 2015 at 16:27 UTC
How did you tell Perl to search for modules in the /usr/local/lib directory? لսႽ† ᥲᥒ⚪⟊Ⴙᘓᖇ Ꮅᘓᖇ⎱ Ⴙᥲ𝇋ƙᘓᖇ	[reply]
Re: Upgrading XS module to later version by locked_user sundialsvc4 (Abbot) on Jun 19, 2015 at 03:22 UTC
If you felt that you had to “hack” anything, you definitely strayed down the wrong path. No such thing should be necessary, and, as you can readily guess, it can get you a lot of trouble. You should be able to treat this like a “install modules as a non-`root` user on shard-hosting” situation, specifying (within ordinary `cpan/cpanm`, and in the usual way) your own “local” package directory and of course your own overrides for `PERL5LIB`. At that point, you should be able to install any package, including `List::Util`, and confirm that the desired copy winds up in that designated place. You should be able to do all of this with no special privileges at all. (You want the system libraries, as maintained by the opsys’s package manager, to remain “untouchable,” as they are.) It should not matter whether the package has any “XS” content or not. Unfortunately, not every package out there correctly specifies the minimum version that it needs for its prerequisites, and some don’t accurately list the prerequisites that they do have. The result is that `cpan/cpanm` does not realize that a later version of something needs to be installed as a pre-/co-requisite. But you ought to be able to manually install it, then repeat the previous installation, and the manually-installed version ought to be properly seen and incorporated.
Re^2: Upgrading XS module to later version by Anonymous Monk on Feb 11, 2022 at 15:33 UTC
I ran into the same problem, when trying to install `cpan Moose`, it says that `Installing the dependencies failed: Installed version (1.55) of List:: +Util is not in range '1.56'` [download] Trying to install List::Util with `cpanm List::Util` seems to work, it show the message `Successfully installed Scalar-List-Utils-1.60 (upgraded from 1.55)` [download] But when then trying to run `cpan Moose` it complains again about error `Installing the dependencies failed: Installed version (1.55) of List:: +Util is not in range '1.56'` [download] Obviously, cpanm did not upgrade List::Utils. but installed it in a different directory. When I check the installed packages, I get this `$ cpan -l \|grep List::Util List::Util 1.55 List::Util::XS 1.55 Module::CoreList::Utils 5.20210520 List::Util 1.55 List::Util::XS 1.55 Module::CoreList::Utils 5.20210520 List::Util 1.55 List::Util::XS 1.55 site_perl::List::Util 1.60 site_perl::List::Util::XS 1.60 List::Util 1.60 List::Util::XS 1.60 List::Util 1.60 List::Util::XS 1.60 List::Util 1.55 List::Util::XS 1.55 Module::CoreList::Utils 5.20210520 List::Util 1.55 List::Util::XS 1.55` [download] It is not clear to me how to ensure that the newer version should used.	[reply] [d/l] [select]
Re^3: Upgrading XS module to later version by marto (Cardinal) on Feb 11, 2022 at 15:44 UTC
do you have multiple perls installed? Are you using perlbrew or local::lib? What does `which cpan` and `which cpanm` show?	[reply] [d/l] [select]
Re^4: Upgrading XS module to later version by Anonymous Monk on Feb 11, 2022 at 16:35 UTC
Re^5: Upgrading XS module to later version by ikegami (Patriarch) on Feb 14, 2022 at 00:55 UTC
Re^3: Upgrading XS module to later version by ikegami (Patriarch) on Feb 14, 2022 at 00:56 UTC
By the way, `export PERL5LIB=$PREFIX/lib` is useless. (This was supposed to be a reply to Re^4: Upgrading XS module to later version.)	[reply] [d/l]
Re^3: Upgrading XS module to later version by ikegami (Patriarch) on Feb 14, 2022 at 00:58 UTC
By the way, `export PERL5LIB=$PREFIX/lib` is useless. (sigh, This was supposed to be an update to Re^3: Upgrading XS module to later version.)	[reply] [d/l]
Re^4: Upgrading XS module to later version by Anonymous Monk on Feb 17, 2022 at 15:41 UTC
Re^5: Upgrading XS module to later version by ikegami (Patriarch) on Feb 17, 2022 at 18:58 UTC