in reply to Re^9: Perl crash during perl_clone
in thread Perl crash during perl_clone

Thx for the code example. It matches what I understood about your caching mechanism.

However, my confusion was why there would be 'multiple' contexts: without any perl_clone, I expected only one perl context now.

Anyhow, I modified my code to match the caching scheme and the results are same as when I 'forced' the context : the first callback is invoked fine, but the 2nd callback fails with "Undefined subroutine &main::0 called at" error. It points to a line that contains a "sleep()" statement in the main body of my perl-script. This however happens, just as the 2nd callback executes call_sv().

I also confirmed from prints that 'saved' context is set properly before invoking the call_sv. Unrelated, but the prints also show the same values for the 'saved' context in the two (actually 3) callbacks in my project.

I am unable to run your code as my machine didnt have Inline installed & I'm now getting compile errors executing your code (_867652.xs:13: error: expected â=â, â,â, â;â, âasmâ or â__attribute__â before âCALLBACKâ etc). However, I'm posting the modified version to give a better picture of how my real project looks & hope it reproduces the error I'm seeing.

I cant thank you enough for your efforts here!!

ps: Ofcourse, in my real project is more involved (multiple callbacks executed in each thread & multiple such threads. callbacks and 'main' do lot more than sleep()) 

#! perl -slw
use strict;
use Inline C => Config => BUILD_NOISY => 1;
use Inline C => <<'END_C',  NAME => '_867652', CLEAN_AFTER_BUILD => 0;


PerlInterpreter *saved1;
SV *callback1;

PerlInterpreter *saved2;
SV *callback2;

VOID CALLBACK cbProc1( DWORD time ) {
//this always returns "0"
    PerlInterpreter *temp = Perl_get_context();

    printf( "C-Callback1: %u\n", time );

//my code has a dTHX; here.  Without it, I get errors that my_perl is 
+undefined
    PERL_SET_CONTEXT( saved1 );
    {
        dSP;
        ENTER;
        SAVETMPS;
        PUSHMARK( SP );
        XPUSHs( sv_2mortal( newSVuv( (UV)time ) ) );
        PUTBACK;

        call_sv( callback, G_DISCARD );

        FREETMPS;
        LEAVE;
    }
    PERL_SET_CONTEXT( temp );
    return;
}

VOID CALLBACK cbProc2( DWORD time ) {
//this always returns "0"
    PerlInterpreter *temp = Perl_get_context();

    printf( "C-Callback2: %u\n", time );

//my code has a dTHX; here.  Without it, I get errors that my_perl is 
+undefined

    PERL_SET_CONTEXT( saved2 );
    {
        dSP;
        ENTER;
        SAVETMPS;
        PUSHMARK( SP );
        XPUSHs( sv_2mortal( newSVuv( (UV)time ) ) );
        PUTBACK;

//the error msg shows up after calling this
        call_sv( callback, G_DISCARD );

        FREETMPS;
        LEAVE;
    }
    PERL_SET_CONTEXT( temp );
    return;
}

void __cdecl thread1( VOID *arg ) {
    while( Sleep( 5000 ), 1 ) {
         printf( "Here in thread1\n" );
         cbProc1( GetTickCount() );
     }
}

void __cdecl thread2( VOID *arg ) {
    while( Sleep( 5000 ), 1 ) {
         printf( "Here in thread2\n" );
         cbProc2( GetTickCount() );
     }
}

void setCallback1( SV* cv ) {
//my code did not have the dTHX here, but adding it made no difference
+ (i.e saved1 the same).
    dTHX;
//saved1 and saved2 are always the same too.
    saved1 = Perl_get_context();
    callback1 = cv;

    // Start callback timer thread
    _beginthread( &thread1, 0, NULL );
    return;
}

void setCallback2( SV* cv ) {
//my code did not have the dTHX here, but adding it made no difference
+ (i.e saved2 is the same).
    dTHX;
    saved2 = Perl_get_context();
    callback2 = cv;

    // Start callback timer thread
    _beginthread( &thread2, 0, NULL );
    return;
}

END_C

$|++;

my cb1_cnt = 0;
sub callback1 {
    print "Timer value in callback1 is: $_[0]";
    cb1_cnt++;
    return;
}

my cb2_cnt = 0;
sub callback2 {
    print "Timer value in callback2 is: $_[0]";
    cb2_cnt++;
    return;
}

setCallback1( 'callback1' );

while (cb1_cnt != 4)
{
  sleep(1);
}

setCallback2('callback2');

while (cb2_cnt != 4)
{
#the error msg will point to this line.  Not sure if its just a
#coincidence that it points here..
  sleep(1);
}

sleep (15);

__END__

[download]

Replies are listed 'Best First'.
Re^11: Perl crash during perl_clone
by BrowserUk (Patriarch) on Oct 29, 2010 at 11:57 UTC

    Your code, with a couple of minor corrections:

    1. Two cases of adding 1 or 2 to the C variable name: call_sv( callback1, G_DISCARD );
    2. Six cases of adding the $ sigil to Perl variable names: my $cb1_cnt = 0;

    runs perfectly here! (Not bad for code you couldn't test :). So, where does that leave us?

    However, my confusion was why there would be 'multiple' contexts: without any perl_clone, I expected only one perl context now.

    If there is only one Perl thread, there will be only one Perl context.

    The problem is, when the callback is triggered and you enter the C-level callback routine (CCB) ready to call the Perl-level callback (PCB), Perl requires that context be established (set), *before* you call the PCB. But, as the CCB is just a C subroutine, that context is not established for you, as it would be with an XS routine. So, you have to do it yourself.

    Hence the need to hive it off somewhere, and establish it before calling back into Perl. "it" here being the global PerlInterpreter *saved(1|2) contexts.

    The local PerlInterpreter *temp contexts within the CCBs are actually redundant in my example code, because I know that the callbacks will only ever be initiated from the C-level timer threads. Ie. they will never be called from an existing Perl context.

    But, and it is a big one, you haven't clarified--nor from memory, even mentioned--what triggers the callbacks in your real application? And this could be very significant.

    For example, on my OS (windows), I know that asynchronous IO callbacks generally run in the context of a completely new thread, started & 'injected' into the Perl process, by the OS. As such, they are clean environments in which we can pretty much do what we like, because as soon as we return from them, they will just disappear with no lasting effects beyond what we program into them.

    However, if you are running on a *nix system, asynchronous events can be triggered through signals. Which means that the CCB will be invoked not in a new, clean thread, but rather in an existing (Perl created) thread, via a longjump. Which means that you need to be very careful what you do to both the global state, and thread-local state of perl during those CCBs. Hence the local save/restore of the temp perl context is my attempt to guess what might be necessary on your platform. (The errors you mention tell me I got it wrong!).

    And this is where my ability to help you curtails rapidly.

    Even if you can provide a clear and concise description of what triggers these callbacks, the chances are I will have little or no experience of those mechanisms on your platform. And even if you could put together some concise, stand-alone code that compiles on your platform and demonstrates the problems you are encountering, it almost certainly wouldn't compile and run here. Which again would leave me in the position of trying to run the code in my head--for an OS I have little knowledge of.

    The bottom line is that without a concise, easily buildable, runnable demonstration of the problem, there is little anyone can really do to help you.

    With such a demo, then you may well find a willing, thread-aware user on your platform (or one sufficiently similar) that can take up the baton. Either here, or perhaps the appropriate CPAN forum or mailing list (See the bottom of the threads POD).

    Such a demo might even draw a response from the most-likely-to-be-able-to-help people, those on p5p. But you need to make it easy for them to try your code and see the problem.

    I'm not abandoning you, just keeping it realistic about my chances of being able to help you.

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]
[select]
      runs perfectly here!

      You must be kidding..Holy cow! Indeed, where does it leave us know..

      Guess, it could be platform-dependent stuff. I'll definitely need to get a demonstration of the problem first. I'd have to learn Inline to get rid of the compile errors (about "CALLBACK" and also that "thread(1|2) are undefined)...or revise the example from my earlier posts. I may pick the latter approach.

      I completely understand your "warnings" on how much you could help, but I feel you will still be quite helpful. So, let me describe a bit more of the project, anyhow.

      We are on *nix platforms (ubuntu and fedora mainly but RH too). The 'arch' is client-server : a standalone executable server (all C). A client-lib (all C) that I've wrapped with SWIG. An application (perl script, in our case) uses the clientlib API to exercise server functionality. Client-server communicate via sockets. Client API spawns threads and sends msgs to server to 'do things'. Server program will use some HW to do its work. When done, server sendsback msgs, results via the socket. The threads will then call the approriate CCBs based on various msgs received from server. Make sense? I hope :-)

      Which means that the CCB will be invoked not in a new, clean thread, but rather in an existing (Perl created) thread, via a longjump

      As in the sample code we exchanged, in my real project too, the threads are created on the C side and not by Perl. So, it seems this italisized statement would not apply, right?

[reply]
        but I feel you will still be quite helpful. So, let me describe a bit more of the project

        Then I'm going to need much more detail about the client API and the perl script that is using it.

        You'd need to considerably expand these two steps of the description:

        Client API spawns threads and sends msgs to server to 'do things'.

        The threads will then call the approriate CCBs based on various msgs received from server.

        Questions that come to mind:

        • One (C) thread per server connection?
        • Does that thread procedure sit in a block loop?
        • One PCB per thread?
        • What does the Perl script do whilst waiting for PCBs to be called?
        • Is there only (ever?) one Perl thread?
        • Can two C-threads attempt to call the same PCB? Concurrently?
        in my real project too, the threads are created on the C side and not by Perl. So, it seems this italisized statement would not apply, right?

        If there is only (ever?) one Perl thread in the client program, then there will only be one Perl context.

        But, in that case, by using dTHX: within the CCBs prior to calling the PCBs, all the context issues should (I believe) be resolved. That is certainly the case in both my test code, and my corrected version of your updates to that code, on my OS.

        The fact that your real code is failing, and your attempts to use my test code fail at compile-time, strongly suggests that your problems distinctly fall into the realms of "platform dependant". Hence, my strong reservations about my ability to help you further.

        I could continue to try and re-create your circumstances based entirely upon your (to date, scant) descriptions, but if the underlying problem in your real code would not manifest itself on my platform, I am never going to be able to re-create your problems here. No matter how much effort I expend.

        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]

In Section Seekers of Perl Wisdom