losintikfos has asked for the wisdom of the Perl Monks concerning the following question:

Hi Guys,

I am trying to create a perl embeded application which expose WWW::Mechanize into my Cython extension project. I compile and link my extension using:

x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I/usr/lib/perl/5.18/CORE -I/usr/include/python3.4m -I/home/losintikfos/git/testproj/env/include/python3.4m -c src/perlxsi.c -o build/temp.linux-x86_64-3.4/src/perlxsi.o -O0 -g3 -Wall -c -fmessage-length=0 -Wl,-E -fstack-protector -L/usr/local/lib -L/usr/lib/perl/5.18/CORE -lperl -ldl -lm -lpthread -lc -lcrypt -D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/lib/perl/5.18/CORE

x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I/usr/lib/perl/5.18/CORE -I/usr/include/python3.4m -I/home/losintikfos/git/testproj/env/include/python3.4m -c src/perlmechanize.c -o build/temp.linux-x86_64-3.4/src/perlmechanize.o -O0 -g3 -Wall -c -fmessage-length=0 -Wl,-E -fstack-protector -L/usr/local/lib -L/usr/lib/perl/5.18/CORE -lperl -ldl -lm -lpthread -lc -lcrypt -D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/lib/perl/5.18/CORE

x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.4/src/perlxsi.o build/temp.linux-x86_64-3.4/src/perlmechanize.o build/temp.linux-x86_64-3.4/src/pymechanize.o -lperl -o build/lib.linux-x86_64-3.4/pymechanize.cpython-34m.so

Which works fine. My problem is - when I import calls from my extension module pymechanize.cpython-34m.so I get below error:

Can't load '/usr/local/lib/perl/5.18.2/auto/Encode/Encode.so' for module Encode: /usr/local/lib/perl/5.18.2/auto/Encode/Encode.so: undefined symbol: PL_utf8skip at /usr/share/perl/5.18/XSLoader.pm line 68.
 at /usr/local/lib/perl/5.18.2/Encode.pm line 10.
Compilation failed in require at /usr/local/share/perl/5.18.2/HTML/Form.pm line 6.
BEGIN failed--compilation aborted at /usr/local/share/perl/5.18.2/HTML/Form.pm line 6.
Compilation failed in require at /usr/local/share/perl/5.18.2/WWW/Mechanize.pm line 133.
BEGIN failed--compilation aborted at /usr/local/share/perl/5.18.2/WWW/Mechanize.pm line 133.
Compilation failed in require.
BEGIN failed--compilation aborted.
Attempt to reload WWW/Mechanize.pm aborted.
Compilation failed in require at (eval 9) line 1.
BEGIN failed--compilation aborted at (eval 9) line 1.

For almost two weeks now I have tried various options - including reinstalling Encode.pm but to no luck. I can see the symbol is there when I do:

nm /usr/local/lib/perl/5.18.2/auto/Encode/Encode.so | grep PL_utf8skip

Do anyone know what might be causing this?

  • Comment on Issue: Encode.so: undefined symbol: PL_utf8skip

Replies are listed 'Best First'.
Re: Issue: Encode.so: undefined symbol: PL_utf8skip
by 1nickt (Canon) on Jul 23, 2015 at 20:25 UTC

    Could be the perl running your program is not the one you used to compile your extension. Check your perl with -v at the command line, and its @INC with -V

    The way forward always starts with a minimal test.
      Thanks @1nickt.

      I have tried:

      perl -e "print \"@INC\""
      And got:
      /etc/perl /usr/local/lib/perl/5.18.2 /usr/local/share/perl/5.18.2 /usr +/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.18 /usr/share/perl/5.18 / +usr/local/lib/site_perl

      and perl -v and got:

      This is perl 5, version 18, subversion 2 (v5.18.2) built for x86_64-li +nux-gnu-thread-multi (with 41 registered patches, see perl -V for more detail)
      Which I think does looks fine.
Re: Issue: Encode.so: undefined symbol: PL_utf8skip
by Anonymous Monk on Jul 23, 2015 at 22:24 UTC

    . I can see the symbol is there when I do:

    No, PL_utf8skip is only used there, its not defined there, PL_utf8skip comes from libperl516.so

    To see how dlls depend on each other use ldd -v ...so / ldd -v ..so like this (also includes other tips)

      Thanks.

      I was hoping that libperl.so located in the standard library directory /usr/lib will get resolved by the link flag -lperl and hence make the symbol PL_utf8skip available.

      ldd -v /usr/local/lib/perl/5.18.2/auto/Encode/Encode.so

      Results:

      	linux-vdso.so.1 =>  (0x00007ffcacdc7000)
      	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7c44f87000)
      	/lib64/ld-linux-x86-64.so.2 (0x00007f7c45555000)
      
      	Version information:
      	/usr/local/lib/perl/5.18.2/auto/Encode/Encode.so:
      		libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
      	/lib/x86_64-linux-gnu/libc.so.6:
      		ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
      		ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
      

        Well, for whatever reason, Encode.so doesn't appear linked to libperl.so ... thats weird, do the other .so's link to libperl? Like pymechanize.o or whatever it is you're generating?

Re: Issue: Encode.so: undefined symbol: PL_utf8skip
by ikegami (Patriarch) on Jul 25, 2015 at 06:40 UTC

      ikegami: Did you follow the instructions at all?

      Do you think XSLoader could run and produce an error message if the instructions weren't followed?

      @ikegami - Yes I did. Thanks.

        I am now looking into an option of explicitly compiling header perl.h directly into my extension library. But I am getting the error below.

        My Error:

        x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I/usr/lib/perl/5.18/CORE -I/usr/include -I/usr/local/include -I/usr/include/python3.4m -I/home/losintikfos/git/testproj/env/include/python3.4m -c src/pymechanize.c -o build/temp.linux-x86_64-3.4/src/pymechanize.o -O0 -g3 -Wall -c -fmessage-length=0 -Wl,-E -fstack-protector -L/usr/local/lib -L/usr/lib/perl/5.18/CORE -lperl -ldl -lm -lpthread -lc -lcrypt -D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/lib/perl/5.18/CORE
        In file included from /usr/lib/perl/5.18/CORE/perl.h:2513:0,
                         from src/pymechanize.c:297:
        /usr/lib/perl/5.18/CORE/handy.h:873:1: error: unknown type name ‘EXTCONST’
         EXTCONST U32 PL_charclass[];
         ^
        /usr/lib/perl/5.18/CORE/handy.h:873:14: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘PL_charclass’
         EXTCONST U32 PL_charclass[];
                      ^
        In file included from /usr/lib/perl/5.18/CORE/regexp.h:21:0,
                         from /usr/lib/perl/5.18/CORE/perl.h:3472,
                         from src/pymechanize.c:297:
        /usr/lib/perl/5.18/CORE/utf8.h:100:10: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘unsigned’
         EXTCONST unsigned char PL_utf8skip[];
                  ^
        In file included from src/pymechanize.c:297:0:
        /usr/lib/perl/5.18/CORE/perl.h:4254:10: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘char’
         EXTCONST char PL_warn_uninit[]
                  ^
        /usr/lib/perl/5.18/CORE/perl.h:4256:10: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘char’
         EXTCONST char PL_warn_uninit_sv[]
                  ^
        /usr/lib/perl/5.18/CORE/perl.h:4258:10: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘char’
         EXTCONST char PL_warn_nosemi[]
                  ^
        /usr/lib/perl/5.18/CORE/perl.h:4260:10: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘char’
         EXTCONST char PL_warn_reserved[]
                  ^
        /usr/lib/perl/5.18/CORE/perl.h:4262:10: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘char
        
        ....
        
        Any ideas monks.