in reply to Re^3: Devel::NYTProf: "undefined symbol: PL_stack_sp" error
in thread Devel::NYTProf: "undefined symbol: PL_stack_sp" error

How do module directories get associated with specific installation locations of perl? When I install a module, I just do the following (as an example): 

cd /home/user/perl_modules
cpanm -L$PWD Devel::NYTProf

[download]

Replies are listed 'Best First'.
Re^5: Devel::NYTProf: "undefined symbol: PL_stack_sp" error
by afoken (Chancellor) on Aug 28, 2021 at 00:14 UTC
    How do module directories get associated with specific installation locations of perl?

    Easy: The library paths are compiled into the perl binary. In case of Linux, most of the library paths are actually in libperl.so, the perl executable is barely more than a stub that contains just enough code to load libperl.so, and the directory where libperl.so can be found:

    /home/alex>cd /usr/bin/

/usr/bin>strings perl | grep /usr
/usr/lib64/perl5/CORE

/usr/bin>ldd perl
        linux-vdso.so.1 (0x00007ffe43ef0000)
        libperl.so => /usr/lib64/perl5/CORE/libperl.so (0x00007f2387f4
+3000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f2387d26000)
        libnsl.so.1 => /lib64/libnsl.so.1 (0x00007f2387b0c000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f2387908000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f23875ff000)
        libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f23873c7000)
        libutil.so.1 => /lib64/libutil.so.1 (0x00007f23871c4000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f2386dfb000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f2388327000)

/usr/bin>strings /usr/lib64/perl5/CORE/libperl.so | grep /usr
/usr/local/lib64/perl5
/usr/local/share/perl5
/usr/lib64/perl5/vendor_perl
/usr/share/perl5/vendor_perl
/usr/lib64/perl5
/usr/share/perl5

/usr/bin>./perl -E 'say for @INC'
/usr/local/lib64/perl5
/usr/local/share/perl5
/usr/lib64/perl5/vendor_perl
/usr/share/perl5/vendor_perl
/usr/lib64/perl5
/usr/share/perl5
.

/usr/bin>ls -l perl perl5.22.2
lrwxrwxrwx 1 root root    10 Jan  8  2017 perl -> perl5.22.2*
-rwxr-xr-x 1 root root 10376 Apr 30  2016 perl5.22.2*

/usr/bin>

    [download]

    The CPAN utilities (or, to be more precise: the module build utilties) install into the directories that were choosen when perl was configured. That information is available via Config, and is actually stored in Config_heavy.pl:

    /usr/bin>grep /usr /usr/lib64/perl5/Config_heavy.pl |grep perl
archlib='/usr/lib64/perl5'
archlibexp='/usr/lib64/perl5'
ccdlflags='-Wl,-E -Wl,-rpath,/usr/lib64/perl5/CORE'
config_arg10='-Dvendorarch=/usr/lib64/perl5/vendor_perl'
config_arg4='-Dsitelib=/usr/local/share/perl5'
config_arg5='-Dsitearch=/usr/local/lib64/perl5'
config_arg6='-Darchlib=/usr/lib64/perl5'
config_arg8='-Dprivlib=/usr/share/perl5'
config_arg9='-Dvendorlib=/usr/share/perl5/vendor_perl'
config_args='-de -Dprefix=/usr -Dsiteprefix=/usr/local -Dsitelib=/usr/
+local/share/perl5 -Dsitearch=/usr/local/lib64/perl5 -Darchlib=/usr/li
+b64/perl5 -Dvendorprefix=/usr -Dprivlib=/usr/share/perl5 -Dvendorlib=
+/usr/share/perl5/vendor_perl -Dvendorarch=/usr/lib64/perl5/vendor_per
+l -Dscriptdir=/usr/bin -Dcccdlflags=-fPIC -Dinstallprefix=/usr -Dlibp
+th=/usr/local/lib64 /usr/lib64 /lib64 -Doptimize=-O2 -fPIC -Dusethrea
+ds -Duseithreads -Duseshrplib -Ubincompat5005 -Uversiononly -Dpager=/
+usr/bin/less -isr -Darchname=x86_64-linux'
installarchlib='/usr/lib64/perl5'
installprivlib='/usr/share/perl5'
installsitearch='/usr/local/lib64/perl5'
installsitelib='/usr/local/share/perl5'
installvendorarch='/usr/lib64/perl5/vendor_perl'
installvendorlib='/usr/share/perl5/vendor_perl'
perlpath='/usr/bin/perl'
privlib='/usr/share/perl5'
privlibexp='/usr/share/perl5'
sitearch='/usr/local/lib64/perl5'
sitearchexp='/usr/local/lib64/perl5'
sitelib='/usr/local/share/perl5'
sitelib_stem='/usr/local/share/perl5'
sitelibexp='/usr/local/share/perl5'
startperl='#!/usr/bin/perl'
vendorarch='/usr/lib64/perl5/vendor_perl'
vendorarchexp='/usr/lib64/perl5/vendor_perl'
vendorlib='/usr/share/perl5/vendor_perl'
vendorlib_stem='/usr/share/perl5/vendor_perl'
vendorlibexp='/usr/share/perl5/vendor_perl'

/usr/bin>

    [download]

    The build modules simply load and query Config, simply by using use or require. Config.pm and Config_heavy.pl are created when perl is compiled, and installed into one of the directories compiled into the binaries. Because each perl has its own set of include directories, simply loading Config.pm is sufficient (unless someone has messed with @INC or the data in Config.pm or Config_heavy.pl - see the warning in Config).

    (All examples from Slackware Linux 14.2 64 bit, using the system perl.)

    Alexander

    --
    Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
[reply]
[d/l]
[select]

      It seems I have multiple sources of confusion here. First of all, the Devel::NYTProf module directory contains the following file (referenced in the error message in my OP): 

      /user/perl_modules/lib/perl5/x86_64-linux/Devel/auto/Devel/NYTProf/NYT
+Prof.so

      [download]

      Which appears to be called using this line in /user/perl_modules/lib/perl5/x86_64-linux/Devel/NYTProf/Core.pm: 

      XSLoader::load('Devel::NYTProf', $VERSION);

      [download]

      Based on a cursory search this appears to be a way to include a library of separate C/C++ functions and access them from within perl. Is that correct? If so, then that probably explains why I've never run into this error until now - other modules I've installed and referenced from /user/perl_modules/lib/perl5 didn't have dynamic libraries.

      Having said that, based on my C/C++ experience linker errors such as "undefined symbol" occur when the executable doesn't have a path to the referenced dynamic library, which can be checked using the "ldd" command. If I run "ldd /usr/bin/perl", it returns the following: 

      > ldd /usr/bin/perl
linux-vdso.so.1 =>  (0x00007ffc0b140000)
libperl.so => /usr/lib64/perl5/CORE/libperl.so (0x00007f74bbaa5000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f74bb88c000)
libnsl.so.1 => /lib64/libnsl.so.1 (0x00007f74bb672000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f74bb46e000)
libm.so.6 => /lib64/libm.so.6 (0x00007f74bb16c000)
libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f74baf35000)
libutil.so.1 => /lib64/libutil.so.1 (0x00007f74bad32000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f74bab16000)
libc.so.6 => /lib64/libc.so.6 (0x00007f74ba749000)
/lib64/ld-linux-x86-64.so.2 (0x00007f74bbe33000)
libfreebl3.so => /lib64/libfreebl3.so (0x00007f74ba546000)

      [download]

      whereas "ldd /tool/bin/perl" returns the following: 

      > ldd /tool/bin/perl
linux-vdso.so.1 =>  (0x00007ffeeebb7000)
libnsl.so.1 => /lib64/libnsl.so.1 (0x00007f3a2cd0f000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f3a2cb0b000)
libm.so.6 => /lib64/libm.so.6 (0x00007f3a2c809000)
libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f3a2c5d2000)
libutil.so.1 => /lib64/libutil.so.1 (0x00007f3a2c002000)
libc.so.6 => /lib64/libc.so.6 (0x00007f3a2c002000)
/lib64/ld-llinux-x86-64.so.2 (0x00007f3a2cf29000)
libfreebl3.so => /lib64/libfreebl3.so (0x00007f3a2bdff000)

      [download]

      Note that neither perl installation has /user/perl_modules/lib/perl5/x86_64-linux/Devel/auto/Devel/NYTProf/ in its list of ldd paths, so how was /tool/bin/perl able to find NYTProf.so but /usr/bin/perl was not?
      Also the ldd paths for a given perl installation are completely separate from the paths contained in @INC when that perl executable is run, correct?

[reply]
[d/l]
[select]

        the Devel::NYTProf module directory contains the following file (referenced in the error message in my OP):

        /user/perl_modules/lib/perl5/x86_64-linux/Devel/auto/Devel/NYTProf/NYT
+Prof.so

        [download]

        Which appears to be called using this line in /user/perl_modules/lib/perl5/x86_64-linux/Devel/NYTProf/Core.pm: 

        XSLoader::load('Devel::NYTProf', $VERSION);

        [download]

        Based on a cursory search this appears to be a way to include a library of separate C/C++ functions and access them from within perl. Is that correct?

        Yes, that's the XS mechanism. (See perlxs for far more information than you want to know now.) The actual C library is loaded by DynaLoader or the simplified XSLoader, and that's where the next part is anwered:

        neither perl installation has /user/perl_modules/lib/perl5/x86_64-linux/Devel/auto/Devel/NYTProf/ in its list of ldd paths, so how was /tool/bin/perl able to find NYTProf.so but /usr/bin/perl was not? Also the ldd paths for a given perl installation are completely separate from the paths contained in @INC when that perl executable is run, correct?

        In case of Devel::NYTProf, the XS library is searched in @INC, not in the system's library path, because Devel::NYTProf uses XSLoader:

        /home/alex>strace -e trace=file -o /tmp/trace perl -MDevel::NYTProf -e
+ 1

/home/alex>grep NYTProf /tmp/trace
execve("/usr/local/bin/perl", ["perl", "-MDevel::NYTProf", "-e", "1"],
+ [/* 39 vars */]) = 0
stat("/usr/local/lib64/perl5/Devel/NYTProf.pmc", 0x7ffce1098b60) = -1 
+ENOENT (No such file or directory)
stat("/usr/local/lib64/perl5/Devel/NYTProf.pm", {st_mode=S_IFREG|0444,
+ st_size=53032, ...}) = 0
open("/usr/local/lib64/perl5/Devel/NYTProf.pm", O_RDONLY) = 4
stat("/usr/local/lib64/perl5/Devel/NYTProf/Core.pmc", 0x7ffce1098b60) 
+= -1 ENOENT (No such file or directory)
stat("/usr/local/lib64/perl5/Devel/NYTProf/Core.pm", {st_mode=S_IFREG|
+0444, st_size=5953, ...}) = 0
open("/usr/local/lib64/perl5/Devel/NYTProf/Core.pm", O_RDONLY) = 4
stat("/usr/local/lib64/perl5/auto/Devel/NYTProf/NYTProf.bs", 0x259d298
+) = -1 ENOENT (No such file or directory)
stat("/usr/local/lib64/perl5/auto/Devel/NYTProf/NYTProf.so", {st_mode=
+S_IFREG|0555, st_size=145632, ...}) = 0
open("/usr/local/lib64/perl5/auto/Devel/NYTProf/NYTProf.so", O_RDONLY|
+O_CLOEXEC) = 4

/home/alex>

        [download]

        What happens is that perl tries to load Devel::NYTProf from files named Devel/NYTProf.pmc or Devel/NYTProf.pm, in all paths in @INC, stopping at the first found. That's normal behaviour of use/require/do. Devel::NYTProf loads Devel::NYTProf::Core, and perl tries to load it from files named Devel/NYTProf/Core.pmc or Devel/NYTProf/Core.pm.

        Then Devel::NYTProf::Core calls XSLoader::load("Devel::NYTProf",...). That's a tiny bit more complicated than shown in XSLoader, where load() is just called as XSLoader::load(__PACKAGE__,...). Usually, Devel::NYTProf would directly call XSLoader, but it seems to have grown so far that it needed to be split in two parts. Not really relevant here.

        What happens is that XSLoader::load() more or less just replaces :: by / in the package name, prefixes auto/, and appends the name of the bootstrap and library files. So it tries to find auto/Devel/NYTProf/NYTProf.bs and auto/Devel/NYTProf/NYTProf.so. It tries to search only where the matching *.pm file was found (which makes sense, because that's where the installation tools will place it), but may revert to searching in @INC. See https://metacpan.org/dist/XSLoader/source/XSLoader.pm.

        Note that XSLoader actually searches the directory where the calling module was loaded from, not the one for the module name passed as argument. Usually, that's the same file, so it usually makes sense. In case of Devel::NYTProf, it looks a little bit confusing because it uses the directory containing Devel/NYTProf/Core.pm instead of the directory containing Devel/NYTProf.pm to load the XS library for Devel::NYTProf, but again, the installation tools placed all of them below the same directory. So it does not really matter and is right in the common case. You usually don't have two completely different modules A and B, installed below different directories, and have module B load the XS for module A or vice versa.

        Alexander

        --
        Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
[reply]
[d/l]
[select]
Re^5: Devel::NYTProf: "undefined symbol: PL_stack_sp" error
by ikegami (Patriarch) on Aug 28, 2021 at 00:08 UTC

    It's not so much associated with a specific install location, but the version and settings of a specific build of perl.

    The perl that was used to install them. (head -n 1 "$( type -p cpanm )" )

    Upd: Added missing -p.

    Seeking work! You can reach me at ikegami@adaelis.com

[reply]
[d/l]
[select]

      type or which?

[reply]
[d/l]
[select]
[reply]
[d/l]
[select]
Re^5: Devel::NYTProf: "undefined symbol: PL_stack_sp" error
by swl (Prior) on Aug 27, 2021 at 23:58 UTC
[reply]

In Section Seekers of Perl Wisdom