but I feel you will still be quite helpful. So, let me describe a bit more of the project

Then I'm going to need much more detail about the client API and the perl script that is using it.

You'd need to considerably expand these two steps of the description:

Client API spawns threads and sends msgs to server to 'do things'.

The threads will then call the approriate CCBs based on various msgs received from server.

Questions that come to mind:

• One (C) thread per server connection?
• Does that thread procedure sit in a block loop?
• One PCB per thread?
• What does the Perl script do whilst waiting for PCBs to be called?
• Is there only (ever?) one Perl thread?
• Can two C-threads attempt to call the same PCB? Concurrently?

in my real project too, the threads are created on the C side and not by Perl. So, it seems this italisized statement would not apply, right?

If there is only (ever?) one Perl thread in the client program, then there will only be one Perl context.

But, in that case, by using dTHX: within the CCBs prior to calling the PCBs, all the context issues should (I believe) be resolved. That is certainly the case in both my test code, and my corrected version of your updates to that code, on my OS.

The fact that your real code is failing, and your attempts to use my test code fail at compile-time, strongly suggests that your problems distinctly fall into the realms of "platform dependant". Hence, my strong reservations about my ability to help you further.

I could continue to try and re-create your circumstances based entirely upon your (to date, scant) descriptions, but if the underlying problem in your real code would not manifest itself on my platform, I am never going to be able to re-create your problems here. No matter how much effort I expend.

I'm traveling and will be limited access to work on the code. I'll respond to your questions now, but would suggest that you do not spend any time besides 'thinking' and asking me further questions/or offer 'gut-feel' ideas.

One (C) thread per server connection?

Yes. Note: Several of the C API can end-up creating independent C threads, each with a separate connection to the server.

Does that thread procedure sit in a block loop?

Yes. Depnding on 'responses' from the server, it will take appropriate actions, including invoking CCB at appropriate times.

One PCB per thread?

Not necessarily.

What does the Perl script do whilst waiting for PCBs to be called?

Not much, Id say. The script typically goes like this : Call an API, go into a "while" loop, waiting on a flag for the expected PCB to complete. PCB sets the flag, passes the results back thru another variables. (Perl HASH wrappers for the underlying C structures). Depending on the results, proceeds with the next action/API until a pass/fail verdict can be produced.

Is there only (ever?) one Perl thread?

There are no perl_clone's anymore. So, only one Perl context created by perl.exe itself. But, you probably didnt mean that by Perl thread: I do not create any perl level threads (itheads?) in the perl script. Its a 'main' & a bunch of helper subs (in a separate .pm) and a bunch of PCBs.

However, I would like, if possible, to allow script developer flexibility to use perl threads in the future.

Can two C-threads attempt to call the same PCB?

No, though there is a many-to-1 relation between PCBs and C-threads, each PCB is handled only by one C-thread.

Concurrently?

Umm..technically it maybe possible : I mean, two C-threads each calling a separate PCB 'concurrently' (i.e a 2nd PCB is invoked before 1st PCB completes). I believe all the C-threads are at the same priority, so its not likely. However, I'm certain there is no such situation occuring with the simple scripts that are giving me problems now.

The system follows an multi-threaded, multi-process, async event-driven programming model. The Perl script built using the wrapped C-API is the driver/initiator of all activities. Each perl script would try to exercise a specific scenario that that could occur in a production app.

I am more than happy to provide further details, but may not be seeing what details will help you. So, appreciate you asking specific question.

Hope this helps and thanks once again!

Hi BrowserUk:

I got back in town and got your original and a modified version working (mostly changes like VOID to void * etc were needed). Ofcourse, this worked just as in your experience. So, didnt post that version here. However, I got two other versions that do not work and fail similar to my real project. I've posted them below. I really hope you will be able to shed some light. Thanks

cback_test.pl & cback_test.pm : In this variant, both CCBs are invoked from same thread. I tried to structure the files closer to how my project files are structured (i.e separate pm). Please see addl comments in the code too.

The 2nd version is posted further below

#! perl -slw
use strict;
use cback_test;
use Inline C => Config => BUILD_NOISY => 1;
use Inline C => <<'END_C',  NAME => '_867652', CLEAN_AFTER_BUILD => 0;

PerlInterpreter *saved1;
SV *cback1;

PerlInterpreter *saved2;
SV *cback2;

void cbProc1( unsigned int time ) {
//this always returns "0"
    PerlInterpreter *temp = Perl_get_context();

    printf( "Time in C-Callback1: %u. Prior ctx %lud\n", time , temp);

    PERL_SET_CONTEXT( saved1 );
    {
        dSP;
        ENTER;
        SAVETMPS;
        PUSHMARK( SP );
        XPUSHs( sv_2mortal( newSVuv( (UV)time ) ) );
        PUTBACK;

        call_sv( cback1, G_DISCARD );

        FREETMPS;
        LEAVE;
    }
    PERL_SET_CONTEXT( temp );
    return;
}

void cbProc2( unsigned int time ) {
//this always returns "0"
    PerlInterpreter *temp = Perl_get_context();

    printf( "Time in C-Callback2: %u.  Prior ctx %lud\n", time , temp)
+;

    PERL_SET_CONTEXT( saved2 );
    {
        dSP;
        ENTER;
        SAVETMPS;
        PUSHMARK( SP );
        XPUSHs( sv_2mortal( newSVuv( (UV)time ) ) );
        PUTBACK;

//the error msg shows up after calling this
        call_sv( cback2, G_DISCARD );

        FREETMPS;
        LEAVE;
    }
    PERL_SET_CONTEXT( temp );
    return;
}

void* thread1( void *arg ) {
    int i=0;
    while( i++<5 ) {
         sleep( 5 );
         printf( "Here in thread1..calling CCB1\n" );
         cbProc1( time(NULL) );
     }
    i=0;
    while( i++<5 ) {
         sleep( 5 );
         printf( "Here in thread1..calling CCB2\n" );
         cbProc2( time(NULL) );
     }
}

void* thread2( void *arg ) {
    while( sleep( 7 ), 1 ) {
         printf( "Here in thread2\n" );
         cbProc2( time(NULL) );
     }
}

void setCallback1( SV* cv ) {
    pthread_t tid;
//my code did not have the dTHX here, but adding it made no difference
+ (i.e saved1 the same).
    //dTHX;
//saved1 and saved2 are always the same too.
    saved1 = Perl_get_context();
    cback1 = cv;

    printf("saved1 = %lud\n", saved1);
    // Start callback timer thread
    pthread_create( &tid, NULL, thread1, NULL );
    return;
}

void setCallback2( SV* cv ) {
    pthread_t tid;
//my code did not have the dTHX here, but adding it made no difference
+ (i.e saved2 is the same).
    //dTHX;
    saved2 = Perl_get_context();
    cback2 = cv;

    printf("saved2 = %lud\n", saved2);
    // Start callback timer thread
    //pthread_create( &tid, NULL, thread2, NULL );
    return;
}

END_C

$|++;

print "Setting callback1..\n";

#Registering cback rhis gives a coredump or one or several other error
+s
#when CCB calls PCB
#setCallback1(\&cback_test::callback1);

#this syntax works as expected in this program, but this doesnt
#work (the pcb ref is not valid or null) when my typemap using SWIG ru
+ns
setCallback1('cback_test::callback1');

while ($cback_test::cb1_cnt != 4)
{
  print "Waiting for callback1 to finish..\n";
  sleep(1);
}

print "Setting callback2..\n";
setCallback2(\&cback_test::callback2);

while ($cback_test::cb2_cnt != 4)
{
  print "Waiting for callback2 to finish..\n";
  sleep(1);
}

sleep (15);

__END__
[download]

package cback_test;

$cb1_cnt = 0;
sub callback1 {
    print "Timer value in callback1 is: $_[0]";
    $cb1_cnt++;
    return;
}

$cb2_cnt = 0;
sub callback2 {
    print "Timer value in callback2 is: $_[0]";
    $cb2_cnt++;
    return;
}

1;
[download]

Here in thread1..calling CCB2
Time in C-Callback2: 1289205658.  Prior ctx 0d
Use of inherited AUTOLOAD for non-method main::1289205658() is depreca
+ted at cback_test3.pl line 137.
Segmentation fault
[download]

myModule.xs and test.pl : These are from the a "fake" module I created using  h2xs -AX myModule. I then had to  ln -s lib/myModule.pm . and  ln -s blib/arch/auto/myModule/myModule.so . to successfully run the test.pl (I'm sure there is a better way, but this worked).

#include "EXTERN.h"
#include "perl.h"
#include "XSUB.h"

#include "ppport.h"

#include <pthread.h>
static PerlInterpreter *orig_perl=NULL;

SV* cb_ptr1 = NULL;
SV* cb_ptr2 = NULL;

void InvokeCB ()
{
   static int val = 0;
   SV * sv;

   val++;
   Perl_set_context(orig_perl);
   printf("curr_ctx is %lud\n", Perl_get_context());
   dSP;
   ENTER;
   SAVETMPS;
   PUSHMARK(SP);

   if (val >= 5)
      sv = cb_ptr2;
   else
      sv = cb_ptr1;

   printf("invoking %lud\n", sv);
   XPUSHs(sv_2mortal(newSViv(val)));
                                                                      
+             PUTBACK;
   call_sv(sv, G_DISCARD);

   FREETMPS;
   LEAVE;
}

void * BGThread(void * dontcare)
{
   while (1)
   {
      sleep(5);
      InvokeCB();
   }
}


MODULE = myModule               PACKAGE = myModule

int
RegisterCB1 (SV *SubRef)
    CODE:
       orig_perl = Perl_get_context();
       printf("orig_ctx is %lud\n", orig_perl);
       pthread_t tid;
       pthread_create(&tid, NULL, BGThread, NULL);
       orig_perl = PERL_GET_CONTEXT;
       cb_ptr1 = SubRef;
       printf("registered %lud\n", cb_ptr1);
       RETVAL = 1;
    OUTPUT:
       RETVAL

int
RegisterCB2 (SV *SubRef)
    CODE:
       cb_ptr2 = SubRef;
       printf("registered %lud\n", cb_ptr1);
       RETVAL = 1;
    OUTPUT:
       RETVAL
[download]

#! /usr/local/bin/perl

use myModule;
use warnings;

$cb_done1 = 0;
$cb_done2 = 0;
@results = ();

sub cb_one
{
  ($value) = @_;
   print "cb_onr called. val received : ", $value, "\n";
   $results[scalar(@results)] = $value;

   if ($value == 5)
   {
      print "cb_done changed to one.\n";
      $cb_done1 = 1;
   }
}

sub cb_two
{
  ($value) = @_;
   print "cb_two called. val received : ", $value, "\n";
   $results[scalar(@results)] = $value;

   if ($value == 10)
   {
      print "cb_done2 changed to one.\n";
      $cb_done2 = 1;
   }
}

print "Registering CB1...\n";

$status = myModule::RegisterCB1(\&main::cb_one);
do
{
   print "Waiting for CB1 to be done...\n";
   sleep (1);
} until ($cb_done1 == 1);

print "CB1 was invoked : $cb_done1\n";
print "results are : @results \n";

$status = myModule::RegisterCB2(\&main::cb_two);
do
{
   print "Waiting for CB2 to be done...\n";
   sleep (1);
} until ($cb_done2 == 1);

print "CB2 was invoked : $cb_done2\n";
print "results are : @results \n";
[download]

Registering CB1...
orig_ctx is 151171080d
registered 151189632d
Waiting for CB1 to be done...
Waiting for CB1 to be done...
Waiting for CB1 to be done...
Waiting for CB1 to be done...
Waiting for CB1 to be done...
curr_ctx is 151171080d
invoking 151189632d
Undefined subroutine &main::1 called at test.pl line 42.
make: *** [test_dynamic] Error 255
[download]