Working with multiple interpreters (my_perl is NULL when embed perl in C ; "dSP" segmentation faults)

perlmonk1729 has asked for the wisdom of the Perl Monks concerning the following question:

I am using SWIG to create perl-wrappers for a C library. This library has its own thread, waits on events over a socket and invokes perl-subroutines as callbacks to these events. I'm getting a crash when the thread invokes the subroutines. The "dSP;" line is crashing (see below).

When the subroutine is called, ofcourse, the main perl script is still running. So, I understand my situation is that of multiple perl interpreters running concurrently (one as part of the perl main/script, the 2nd used when the library thread invokes a perl sub)

I have read perlembed, perlguts etc and I believe I have a perl version that supports multiplicity, ithreads and my C library is built with similar options. So, not sure why it is crashing.

As this is all generated code, I'm a unable to figure out what I'm doing wrong. Is my perl not built with the required features? Is there a better way to write the C code below (rather than using constructs like dTHX, dSP etc)?

If I fake invoking the same per-sub callback as part of an C API invoked from the perl script, it works perfectly!

core dump from gdb

entering perl callback handler...
my_perl == NULL

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb7b55b90 (LWP 28603)]
0xb7b6b1e8 in SWIG_Wrap_callback (event=int,
    p_data=0xb7b553a0)
    at ../../../../../..//client/perl/bsa_api_wrap.c:2005
2005          printf("Destruct level : %d...\n", PL_perl_destruct_leve
+l);
(gdb) where
#0  0xb7b6b1e8 in SWIG_Wrap_bsa_disc_callback (event=int,
    p_data=0xb7b553a0)
    at ../../../../../../client/perl/bsa_api_wrap.c:2005
#1  0xb7bce6e5 in bsa_mgt_disc_hdlr (p_msg=0xb7bf3aac)
    at ../../../../../..//client/mgt/bsa_mgt_int.c:63
#2  0xb7bce8ce in callback_task (param=0)
    at ../../../../../..//client/mgt/bsa_mgt_int.c:159
#3  0xb7ec94fb in start_thread () from /lib/tls/i686/cmov/libpthread.s
+o.0
#4  0xb7e4be5e in clone () from /lib/tls/i686/cmov/libc.so.6
[download]

See the C code that calls a Perl subroutine & the compile related options.

void SWIG_Wrap_callback(int event, tMy_MSG *p_data)
{
      printf("entering perl callback handler...\n");
      dTHX;  /*Not sure this is needed, but doesnt make any difference
+ */
      if ( my_perl != NULL )
         printf ("my_perl == %ul\n", my_perl);
      else
         printf ("my_perl == NULL\n");  /* This line is printed*/

      PERL_SET_CONTEXT(my_perl); /* Not sure this is needed, but tried
+ in vain */

      printf("dTHX done...\n");
      dSP;  /* Crashes here */

      printf("Destruct level : %d...\n", PL_perl_destruct_level);  /* 
+"0" (ZERO) is printed under the scenario where callback invokation wo
+rks.  Does it mean perl is not built correctly per my needs?? */

      printf("DSP done...\n");
      SV * sv = bsa_perl_disc_callback;
      if (sv == (SV*)NULL)
         croak("Internal error...\n");

      printf("Preparing to call perl callback...\n");
      PUSHMARK(SP);
      XPUSHs(sv_2mortal(newSViv(event)));
      printf("newSViv of %d done...\n", event);
      XPUSHs(SWIG_NewPointerObj(SWIG_as_voidptr(p_data), SWIGTYPE_p_tM
+y_MSG, SWIG_SHADOW));
      printf("newSVsv...\n");
      PUTBACK;

      printf("calling the Perl sub...\n");
      /* Call the Perl sub */
      call_sv(sv, G_DISCARD);
      printf("Perl sub completed successfully...\n");
}
[download]

The pre-processor output looks like this..

      printf("entering perl callback handler...\n");
      register PerlInterpreter *my_perl __attribute__((unused)) = ((Pe
+rlInterpreter *)pthread_getspecific((*Perl_Gthr_key_ptr(((void *)0)))
+));
      if ( my_perl != ((void *)0) )
         printf ("my_perl == %ul\n", my_perl);
      else
         printf ("my_perl == NULL\n");
      (void)( { int _eC_; if ((_eC_ = pthread_setspecific((*Perl_Gthr_
+key_ptr(((void *)0))), (void *)(my_perl)))) Perl_croak_nocontext("pan
+ic: pthread_setspecific (%d) [%s:%d]", _eC_, "../../../../../../3rdpa
+rty/embedded/brcm/bsa/client/perl/bsa_api_wrap.c", 2005); } );
      printf("dTHX done...\n");
      register SV **sp = (*Perl_Tstack_sp_ptr(((PerlInterpreter *)pthr
+ead_getspecific((*Perl_Gthr_key_ptr(((void *)0)))))));
      printf("Destruct level : %d...\n", (*Perl_Iperl_destruct_level_p
+tr(((PerlInterpreter *)pthread_getspecific((*Perl_Gthr_key_ptr(((void
+ *)0))))))));
      printf("DSP done...\n");
[download]

compilation options used for building my C library

-Dbool=char -fpic -D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEB
+IAN -fno-strict-aliasing -pipe -isystem /usr/local/include -D_LARGEFI
+LE_SOURCE -D_FILE_OFFSET_BITS=64 -DSWIG -g -DMULTIPLICITY -DPERL_IMPL
+ICIT_CONTEXT -DTHREADS_HAVE_PIDS -DUSE_LARGE_FILES -DUSE_PERLIO -DUSE
+_REENTRANT_API -DBSA_CLIENT
[download]

perl -V

Summary of my perl5 (revision 5 version 8 subversion 8) configuration:
  Platform:
    osname=linux, osvers=2.6.24-19-server, archname=i486-linux-gnu-thr
+ead-multi
    uname='linux palmer 2.6.24-19-server #1 smp sat jul 12 00:40:01 ut
+c 2008 i68
6 gnulinux '
    config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -Dccc
+dlflags=-f
PIC -Darchname=i486-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/
+5.8 -Darch
lib=/usr/lib/perl/5.8 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5
+ -Dvendora
rch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/
+perl/5.8.8
 -Dsitearch=/usr/local/lib/perl/5.8.8 -Dman1dir=/usr/share/man/man1 -D
+man3dir=/u
sr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr
+/local/man
/man3 -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uaf
+s -Ud_csh
-Ud_ualarm -Uusesfio -Uusenm -Duseshrplib -Dlibperl=libperl.so.5.8.8 -
+Dd_dosuid
-des'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define usemulti
+plicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS 
+-DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE
+_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN 
+-fno-strict-aliasing -pipe -I/usr/local/include'
    ccversion='', gccversion='4.2.3 (Ubuntu 4.2.3-2ubuntu7)', gccosand
+vers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=1
+2
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
+ lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=/lib/libc-2.7.so, so=so, useshrplib=true, libperl=libperl.so.
+5.8.8
    gnulibc_version='2.7'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib'


Characteristics of this binary (from libperl):
  Compile-time options: MULTIPLICITY PERL_IMPLICIT_CONTEXT
                        PERL_MALLOC_WRAP THREADS_HAVE_PIDS USE_ITHREAD
+S
                        USE_LARGE_FILES USE_PERLIO USE_REENTRANT_API
  Built under linux
  Compiled at Jan 14 2009 22:34:36
  @INC:
    /etc/perl
    /usr/local/lib/perl/5.8.8
:
[download]

Comment on Working with multiple interpreters (my_perl is NULL when embed perl in C ; "dSP" segmentation faults) Select or Download Code

Replies are listed 'Best First'.
Re: my_perl is NULL when embed perl in C ; "dSP" segmentation faults by zwon (Abbot) on Nov 02, 2009 at 19:05 UTC
Do I understand correctly that you creating new thread inside your library? If yes, then probably that's the source of the problem. When perl creates new thread it creates copy of interpreter, so every thread works with its own copy. After thread exits the memory is freed. If new thread is created in C library, the copy of the interpreter isn't created. I'm not too familiar with perl internals, but that's not good.	[reply]
Re^2: my_perl is NULL when embed perl in C ; "dSP" segmentation faults by perlmonk1729 (Acolyte) on Nov 04, 2009 at 04:15 UTC
Thanks Zwon, I had not created any interpreter (perl_alloc) in the thread earlier. I now tried this and it still fails/seg faults inside call_sv(). I'm still trying to understand if I need perl_parse and if I should do a perl_clone rather than perl_alloc. Also, unclear if there is any "locking"/mutual exclusion between the two interpreters required before invoking call_sv Are there any other forums with more experience in this that I should post at? void mythread_fn(void) { static PerlInterpreter my_perl=NULL; my_perl = perl_alloc(); PERL_SET_CONTEXT(my_perl); perl_construct(my_perl); perl_run(my_perl); while (1) { / monitor for events / if (event == xyz) callback_handler(event); } perl_destruct(my_perl); perl_free(my_perl); } void callback_handler(int event) { dTHX; dSP; ENTER; SAVETMPS; SV sv = disc_callback; /* saved earlier in an API call / if (sv == (SV)NULL) croak("Internal error...\n"); PUSHMARK(SP); XPUSHs(sv_2mortal(newSViv(event))); PUTBACK; /* Call the Perl sub / call_sv(sv, G_DISCARD); / crashes as shown below in gdb */ FREETMPS; LEAVE; } Part of gdb stacktrace. #0 Perl_pp_entersub (my_perl=0x6587b0) at pp_hot.c:2945 #1 0x00000000004224f3 in S_call_body (my_perl=0x6587b0, myop=0x65a6e0 +, is_eval=1 '\001') at perl.c:2728 #2 0x00000000004223c4 in Perl_call_sv (my_perl=0x6587b0, sv=0x635420, + flags=2) at perl.c:2607 [download]	[reply] [d/l]