Perl modules not linking to libperl.so

buckrogers1965 has asked for the wisdom of the Perl Monks concerning the following question:

The problem with perl is as follows. All the .so files from the modules built under perl need to have a dependency on libperl.so to properly load into our framework. We would like a way to just find a way to link to libperl.so from our embedded program that just works. I doubt there is a solution this simple, as I have spent a couple of weeks trying every variation of compiling our loadable embedded perl module.

Failing getting our program to "just work" we need to modify perl to support adding a '-lperl' compile flag by default and notify all the distributions to upgrade to this version so that we can get our product out on Linux. This change has no impact on running things using the normal perl command that I saw in my testing.

Failing that as well I can just write a script to walk through the installed perl and "fix" all the .so files to "do the right thing." (TM) But I would rather not do this as it seems very fragile and unmaintainable.

If anyone has any input into how I should proceed from this point, please share.

We have a program that uses embedded perl as a scripting language. If we just embedded perl into the main executable, then everything would work just fine: the libperl.so would be loaded into the global name space of the main program and all subsequent program loads of the various perl submodules (such as DBI.so) would just work.

Unfortunately, we don't embed perl this way, because our program is a framework that loads in a list of modules from disk. This allows us to be extrodinarily flexible in our design, but appears to cause a problem with respect to perl.

An issue that I ran into with porting our program to Linux is that the loader on Linux seems to have a very nice name space for loading libraries in. This allows us to load in libraries attached to the individual loadable modules.

This means that when we load in the perlscript widget, which interfaces to perl and ourselves, then the libperl.so is loaded into the name space of this loadable module.

Loading libraries in this way allows them to also be unloaded easily when the dlopen'ed file is unloaded later.

When we attempt to use additional perl modules in a perl script using something like : "use DBI;"

We get the following error:

     9739:     file=/usr/local/lib/perl5/site_perl/5.9.2/i686-linux/au
+to/DBI/DBI.so;  generating link map
     9739:       dynamic: 0x41e44818  base: 0x41e2f000   size: 0x00015
+be0
     9739:         entry: 0x41e31e20  phdr: 0x41e2f034  phnum:        
+  4
     9739:
     9739:     /usr/local/lib/perl5/site_perl/5.9.2/i686-linux/auto/DB
+I/DBI.so: error: relocation error: undefined symbol: PL_dowarn (fatal
+)
     9739:
     9739:     calling fini: /usr/local/lib/perl5/site_perl/5.9.2/i686
+-linux/auto/DBI/DBI.so
     9739:
Perl error in: ?/Vnos_DB-1
Can't load '/usr/local/lib/perl5/site_perl/5.9.2/i686-linux/auto/DBI/D
+BI.so' for module DBI: /usr/local/lib/perl5/site_perl/5.9.2/i686-linu
+x/auto/DBI/DBI.so: undefined symbol: PL_dowarn at /usr/local/lib/perl
+5/5.9.2/i686-linux/DynaLoader.pm line 201.
at /usr/local/lib/perl5/site_perl/5.9.2/i686-linux/DBI.pm line 254
BEGIN failed--compilation aborted at /usr/local/lib/perl5/site_perl/5.
+9.2/i686-linux/DBI.pm line 254.
Compilation failed in require at (eval 13) line 16.
BEGIN failed--compilation aborted       (in cleanup) Can't load '/usr/
+local/lib/perl5/site_perl/5.9.2/i686-linux/auto/DBI/DBI.so' for modul
+e DBI: /usr/local/lib/perl5/site_perl/5.9.2/i686-linux/auto/DBI/DBI.s
+o: undefined symbol: PL_dowarn at /usr/local/lib/perl5/5.9.2/i686-lin
+ux/DynaLoader.pm line 201.
at /usr/local/lib/perl5/site_perl/5.9.2/i686-linux/DBI.pm line 254
BEGIN failed--compilation aborted at /usr/local/lib/perl5/site_perl/5.
+9.2/i686-linux/DBI.pm line 254.
Compilation failed in require at
Perl error in: ?/Perl Script-1
[download]

The first part is the output from the loader turned on with the "export LD_DEBUG=files" command.

You can see that there is an unresolved symbol in the DBI.so file.

It took me days to track down this issue to the fact that even though the DBI.so file has a dependency on libperl.so, it is not linked against this library.

I ran the command ldd DBI.so

And got the output:

       libc.so.6 => /lib/libc.so.6 (0x40026000)
       /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000)
[download]

Which showed me my problem. Even though DBI.so has symbols that can only be resolved by the libperl.so, it doesn't actually have a dependency on the libperl.so library.

This is bad. According to Ulrich Drepper in his paper on "How to Write Shared Libraries"

"For reasons explained later it is always highly advised to create dependencies with all the DSO's necessary to resolve all references"

I resolved the issue for this one implementation of perl by running the following set of commands on the DBI.so:

cd /usr/local/lib/perl5/site_perl/5.9.2/i686-linux/auto/DBI/

mv DBI.so  test.so

gcc -shared -Wl,-soname,DBI.so -Wl,-rpath,/usr/local/lib/perl5/site_pe
+rl/5.9.2/i686-linux/auto/DBI -o DBI.so /usr/local/lib/perl5/5.9.2/i68
+6-linux/CORE/libperl.so test.so
[download]

Now running this command: ldd DBI.so

Gives us this output:

       /usr/local/lib/perl5/5.9.2/i686-linux/CORE/libperl.so => /usr/l
+ocal/lib/perl5/5.9.2/i686-linux/CORE/libperl.so (0x40003000)
       test.so => /usr/local/lib/perl5/site_perl/5.9.2/i686-linux/auto
+/DBI/test.so (0x400fb000)
       libc.so.6 => /lib/libc.so.6 (0x40121000)
       libnsl.so.1 => /lib/libnsl.so.1 (0x40254000)
       libdl.so.2 => /lib/libdl.so.2 (0x40269000)
       libm.so.6 => /lib/libm.so.6 (0x4026c000)
       libcrypt.so.1 => /lib/libcrypt.so.1 (0x4028e000)
       libutil.so.1 => /lib/libutil.so.1 (0x402bb000)
       /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000)
[download]

Which then results in the following output from the loader when loaded into our program:

     9758:     file=/usr/local/lib/perl5/site_perl/5.9.2/i686-linux/au
+to/DBI/DBI.so;  generating link map
     9758:       dynamic: 0x41e307b8  base: 0x41e2f000   size: 0x00001
+8ec
     9758:         entry: 0x41e2f620  phdr: 0x41e2f034  phnum:        
+  4
     9758:
     9758:
     9758:     file=test.so;  needed by /usr/local/lib/perl5/site_perl
+/5.9.2/i686-linux/auto/DBI/DBI.so
     9758:     file=test.so;  generating link map
     9758:       dynamic: 0x41e46818  base: 0x41e31000   size: 0x00015
+be0
     9758:         entry: 0x41e33e20  phdr: 0x41e31034  phnum:        
+  4
     9758:
     9758:
     9758:     file=/usr/local/lib/perl5/5.9.2/i686-linux/CORE/libperl
+.so;  needed by /usr/local/lib/perl5/site_perl/5.9.2/i686-linux/auto/
+DBI/test.so(relocation dependency)
     9758:
     9758:
     9758:     calling init: /usr/local/lib/perl5/site_perl/5.9.2/i686
+-linux/auto/DBI/test.so
     9758:
     9758:
     9758:     calling init: /usr/local/lib/perl5/site_perl/5.9.2/i686
+-linux/auto/DBI/DBI.so
     9758:
     9758:     opening file=/usr/local/lib/perl5/site_perl/5.9.2/i686-
+linux/auto/DBI/DBI.so; opencount == 1
     9758:
     9758:     file=/usr/local/lib/perl5/5.9.2/i686-linux/auto/Socket/
+Socket.so;  generating link map
     9758:       dynamic: 0x41e487b8  base: 0x41e47000   size: 0x00001
+8ec
     9758:         entry: 0x41e47620  phdr: 0x41e47034  phnum:        
+  4
     9758:
     9758:
     9758:     file=testxxx.so;  needed by /usr/local/lib/perl5/5.9.2/
+i686-linux/auto/Socket/Socket.so
     9758:     file=testxxx.so;  generating link map
     9758:       dynamic: 0x41e4e57c  base: 0x41e49000   size: 0x00005
+744
     9758:         entry: 0x41e4a070  phdr: 0x41e49034  phnum:        
+  4
     9758:
     9758:
     9758:     file=/usr/local/lib/perl5/5.9.2/i686-linux/CORE/libperl
+.so;  needed by /usr/local/lib/perl5/5.9.2/i686-linux/auto/Socket/tes
+txxx.so (relocation dependency)
     9758:
     9758:
     9758:     calling init: /usr/local/lib/perl5/5.9.2/i686-linux/aut
+o/Socket/testxxx.so
     9758:
     9758:
     9758:     calling init: /usr/local/lib/perl5/5.9.2/i686-linux/aut
+o/Socket/Socket.so
     9758:
     9758:     opening file=/usr/local/lib/perl5/5.9.2/i686-linux/auto
+/Socket/Socket.so; opencount == 1
     9758:
[download]

Everything is all happy and good and all linked in.

The problem that I have is that I need to "fix" perl in order for me to successfully support perlscript in our framework.

The problem with perl is the following. All the .so files from the modules built under perl need to have a dependency on libperl.so to properly load into our framework. We need to modify perl to support this by default and notify all the distributions to upgrade to this version so that we can get our product out on Linux. This change has no impact on running things using the normal perl command that I saw in my testing.

So, how do we go about fixing this issue to everyones satisfaction?

Edit planetscape - added <readmore> tags

Comment on Perl modules not linking to libperl.so Select or Download Code

Replies are listed 'Best First'.

Re: Perl modules not linking to libperl.so
by PodMaster (Abbot) on Oct 05, 2005 at 02:50 UTC

... So, how do we go about fixing this issue to everyones satisfaction?

Probably the best way to do it would be by editing Makefile.SH (or whatever is ultimately responsible for setting lddlflags in Config for your platform), so that the core modules are also linked to libperl.so.

update: You could also set the env variable LDLOADLIBS.

On Win32, this is already done through ExtUtils::MakeMaker by setting PERL_ARCHIVE (PERL_ARCHIVE = $(PERL_INC)\perl58.lib), which you could also set by editing ExtUtils::MM_Unix (sub init_linker).

It might be a good idea for ExtUtils::MM_Unix to do this by default, so you might want to contact the perl5-porters and inquire about it.

MJD says "you can't just make shit up and expect the computer to know what you mean, retardo!"
I run a Win32 PPM repository for perl 5.6.x and 5.8.x -- I take requests (README).
** The third rule of perl club is a statement of fact: pod is sexy.

[reply]

Re^2: Perl modules not linking to libperl.so

by buckrogers1965 (Initiate) on Oct 07, 2005 at 22:14 UTC

lddlflags='-shared -L/user/local/lib'

to look like this:

lddlflags='-shared -L/user/local/lib -L/user/local/lib/perl5/5.9.2/i686-linux-thread-multi/CORE -Wl,-rpath,/usr/local/lib/perl5/5.9.2/i686-linux-thread-multi/CORE/ -lperl'

And this does work. Except that I have to do the `make;make install` of Perl first, then modify the line and do `make;make install` a second time to get everything properly installed.

But then once I do that I can build the DBI module and it just works under our framework.

So I think I am very close to a possible small fix to get this working correctly. :D

If anyone has any hints about what I am doing wrong with the lddflags that I have to do the install twice with different settings, please share your thoughts.

I am going to keep looking into this and check out the update about LDLOADLIBS early next week.

Thanks so much!

[reply]