neuty has asked for the wisdom of the Perl Monks concerning the following question:

My department's production servers (RHEL5) have no access to the Internet or any CPAN repository. Therefore, when we need to use a specific Perl module with our scripts, we download them to our terminals from the CPAN website, scp them over to our dev machines, untar them, and then build them locally. We then distribute them to a shared network file system so that other developers may use them, and so that they are available on the prod machines as well. In this way we build a library of Perl modules that slowly grows over time. Our current set of modules were built using my system Perl, which /usr/bin/perl -V lists as revision 5 version 10 subversion 1.

However, we're interested in building a separate copy of Perl that we can also upload to the networked file system. This way we are not tied to using each of our machines' system Perl, which can change from machine to machine (and is frequently 5.8.8, which is outdated for a few necessary modules). A sort of "central" Perl that can be standard across each machine.

Therefore, I needed to build a local copy of Perl in my home directory. I made sure to install the same version as my system Perl, perl-5.10.1. I did this using the following Configure followed by a simple make,make test, make install:

./Configure -des -Dprefix=/home/myuser/localperl -Duserelocatableinc

-Dprefix installed it to the folder localperl in my home directory, while the -Duserelocatableinc flag made it so the @INC would change if the folder was moved (ie., uploaded to the networked file system).

The issue comes when I try to test it out with one of my scripts that requires a module from our library of manually built Perl modules. I keep getting the following error:

/home/myuser/localperl/bin/perl: symbol lookup error: .../auto/DBI/DBI.so: undefined symbol: Perl_Istack_sp_ptr

Obviously my script uses the shared DBI module but is running into errors. From doing research on the Internet, people have said that I would need to recompile the modules using the new build of Perl.

However, the two Perls are the same version and were built on the same machine. I have tried to rebuild the local Perl using the Configure settings as close to the machine Perl as possible to try to see if that would work:

-Dversion=5.10.1
-Dmyhostname=localhost
-Dcc=gcc
-Dinc_version_list=5.10.0
-Darchname=x86_64-linux-thread-multi
-Dusethreads
-Duseithreads
-Duselargefiles

So far this has not stopped the error. My system Perl actually used many more Configure options, but since I was so unfamiliar with them, I didn't throw them into the local Perl build for obvious precautionary reasons.

First, can someone help me get a better understanding of what is preventing these modules from working with my locally built Perl? It makes sense that modules would not work under different versions of Perl, but it surprises me that it doesn't work with the same version built on the same machine. It also does not make sense to me that a module would lock itself info a specific Perl build like a "license" type scenario, so my intuition is that the module is incompatible with some unseen setting/library that the local Perl is built under.

Second, is there a way to get my locally built Perl to work with the modules compiled with the system Perl, or will I have to recompile them all and redistribute a fresh set of modules?

Thank you for any and all help! I have tried Googling the answer, but any issues dealing with this only bring up different Perl versions, which is not the situation I am currently facing.

UPDATE AND SOLUTION

First off, thank you to all those who responded. I wanted to say definitely that YES, you can re-use modules compiled with different builds of Perl that have the same version number. The trick, as stated above, is to build your localperl with the ./Configure flags as close as possible to your system perl. You can determine the ./Configure flags your system Perl uses by running perl -V. You should then use those settings with the flags listed above. My intuition is that -Dcc= and -Darchname= are the critical ones to match, though I have not had time to test this assumption.

So, what was my problem if I had already done all of this, and it still wasn't working? Simple - the test script! In my shebang line, I was still pointing to a faulty build of Perl that I had compiled earlier that day! D'oh! Once I pointed the script to the new build with all of the extra ./Configure options, it worked perfectly fine and was able to use all of the modules built with system Perl!

So, I guess I spent a lot of time trying to put this whole question together. But at least now the Internet (and Perl Monks) will have a definitive answer if others have the same problem.

  • Comment on Can I re-use modules compiled with a different build of Perl, but with the same version number on the same machine?
  • Select or Download Code

Replies are listed 'Best First'.
Re: Can I re-use modules compiled with a different build of Perl, but with the same version number on the same machine?
by syphilis (Archbishop) on Jan 30, 2015 at 23:56 UTC
    From doing research on the Internet, people have said that I would need to recompile the modules using the new build of Perl

    That should fix the problem - but it shouldn't be necessary if the 2 perls in question match.
    I would've expected that if both perls report the same version number, and the same archname then you shouldn't be experiencing this problem ... but maybe there's some other consideration.
    For the 2 perls what is the output of
    perl -le 'print $];'
    and
    perl -V:archname
    If the outputs for the 2 perls are identical, then it's probably best to give us (in readmore tags) the entire perl -V output for both perls.
    Someone might then be able to spot the snag.

    Cheers,
    Rob

      I would've expected that if both perls report the same version number, and the same archname then you shouldn't be experiencing this problem

      It turns out, you were 100% correct. The local Perl built using the same version and specified archname (using the ./Configure flags mentioned above in my post) did fix the problem. The real issue it turns out was the test script that was pointing to a faulty build of Perl.

Re: Can I re-use modules ... ( Crossposted )
by LanX (Saint) on Jan 31, 2015 at 13:35 UTC
      Crap, I had no idea this was considered impolite! I'll watch what I do in the future!
Re: Can I re-use modules compiled with a different build of Perl, but with the same version number on the same machine?
by codiac (Beadle) on Jan 31, 2015 at 09:22 UTC
    Therefore, when we need to use a specific Perl module with our scripts, we download them to our terminals from the CPAN website, scp them over to our dev machines, untar them, and then build them locally.

    Mock + cpanspec is often a lot easier. You can build RHEL5 binary RPMs on any RHEL/Fedora and just scp them over, or pop then in a yum repo on the separate network.

    If you hate manually packaging missing deps, you will love my megadeps patch for mock https://github.com/jfearn/mock

    Basically install megadeps mock + patched cpanspec, set-up a RHEL/CentOS-5 mock root, and assuming your app has a sane spec file, just make the srpm and build that in the mock root with --megadeps, then wait for all the RPMs to be baked.

    (and is frequently 5.8.8, which is outdated for a few necessary modules)

    If it's core modules that need updating, then a lot of them can be made in to RPMs you just need to tweak the spec file so they get installed on a different path. The patched cpanspec linked to on my megadeps mock page automates this tweak.

    Best part is when you upgrade to a newer RHEL/Centos you just set-up a RHEL/Centos-6 mock root and push your applications srpm in to that mock root, and megadeps will only package up the deps missing on that Perl :D

    Of course if you actually need features of a newer Perl you are out of luck :)

    oh! You could look in to using software collections, I believe there is an SCL for Perl 5.16 avaialle for RHEL-5. cpanspec would need some tweaks to spec file it emits, and the mock config would need to have the software collection repos added, but it shouldn't be a lot of work.

Re: Can I re-use modules compiled with a different build of Perl, but with the same version number on the same machine?
by Anonymous Monk on Jan 30, 2015 at 22:03 UTC

    Can I re-use modules compiled with a different build of Perl, but with the same version number on the same machine?

    Maybe, if the two versions are compatible and you work hard enough, but

    if you're compiling a new perl, forget about compatibility with the old version, get a newer version, at least 5.14

    its easier just to use a portable perl like http://www.citrusperl.com/helpweb/installingcitrusperl.htm or http://dwimperl.com/linux.html

      It would be nice to upgrade to the latest version of Perl, but as stated in my post, it would require us to manually rebuild and redistribute every module built with our system Perl, which is currently 5.10.1. So that option was sort of out of the question for us.
Re: Can I re-use modules compiled with a different build of Perl, but with the same version number on the same machine?
by trippledubs (Deacon) on Jan 31, 2015 at 03:58 UTC

    This is -- the worst part of using Linux / Unix. I've found Perl itself really easy to compile, but unless you have active internet connection to CPAN, very time consuming to get everything working right. If you get a nice packaged solution post it! I used a workstation with internet to compile everything and then got it moved, but it took lot of work to get it working and not an easy upgrade. I think an easier route to go is, assuming your machine has all necessary compilers, get the perl source and get a minicpan, though perlbrew might also work. In addition to the other suggestions maybe try this

    bash# strace perl -e 'use DBI;' 2>&1 | grep -i DBI.so stat64("/usr/local/lib/perl/5.18.2/auto/DBI/DBI.so", {st_mode=S_IFREG| +0555, st_size=391823, ...}) = 0 open("/usr/local/lib/perl/5.18.2/auto/DBI/DBI.so", O_RDONLY|O_CLOEXEC) + = 5 bash# ldd -v /usr/local/lib/perl/5.18.2/auto/DBI/DBI.so linux-gate.so.1 => (0xb7727000) libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xb7539000) /lib/ld-linux.so.2 (0xb7728000) Version information: /usr/local/lib/perl/5.18.2/auto/DBI/DBI.so: libc.so.6 (GLIBC_2.3.4) => /lib/i386-linux-gnu/libc.so +.6 libc.so.6 (GLIBC_2.1.3) => /lib/i386-linux-gnu/libc.so +.6 libc.so.6 (GLIBC_2.4) => /lib/i386-linux-gnu/libc.so.6 libc.so.6 (GLIBC_2.0) => /lib/i386-linux-gnu/libc.so.6 libc.so.6 (GLIBC_2.3) => /lib/i386-linux-gnu/libc.so.6 /lib/i386-linux-gnu/libc.so.6: ld-linux.so.2 (GLIBC_2.3) => /lib/ld-linux.so.2 ld-linux.so.2 (GLIBC_PRIVATE) => /lib/ld-linux.so.2 ld-linux.so.2 (GLIBC_2.1) => /lib/ld-linux.so.2

    Maybe that will shed some light on what is going on. You might have to track down all the .so files to make sure they are in the right place. Possibly you need to symlink a .so from somewhere else, or possibly someone else has symlinked a different version. As you can probably already tell, all in the weeds from here.

      If you get a nice packaged solution post it!

      Hey, I just updated my post above with the issue. It turns out matching the ./Configure flags to the system Perl was indeed the solution. The problem was that I was improperly testing the new build with a script that was pointing to an older, faulty build of Perl. Pointing the script to the new build with all of the ./Configure flags properly done corrected all of the issues.

      Also, I appreciate the sympathy for lack of Internet. Lots of people simply reply "That's a dumb solution, why aren't you doing this and this instead" without taking into consideration the limitations that the programmer is forced to deal with as part of the job.

Re: Can I re-use modules compiled with a different build of Perl, but with the same version number on the same machine?
by MidLifeXis (Monsignor) on Feb 02, 2015 at 15:17 UTC

    You could create a local cpan mirror.

    --MidLifeXis

Re: Can I re-use modules compiled with a different build of Perl, but with the same version number on the same machine?
by rmcgowan (Sexton) on Feb 03, 2015 at 00:46 UTC

    It is possible your new build is using DBI but not finding some other required module or shared library, such as the DBD module?

    You could use ldd path/to/DBI.so to see what is not being found for linking, which may give you a hint about where to focus your debugging work.