in reply to Re^2: Perl crash during perl_clone
in thread Perl crash during perl_clone

except that the the coderef was passed into C code from Perl's "default" interpreter (the instance created automatically when I run perl.exe) & NOT from another interpreter/thread spawned by Perl (not sure the difference matters).

What you've described is exactly what I was trying and failing to describe. Two interpreters, one created by perl, one by your C code. And passing a code reference between those two. Perl will not allow you to attempt to pass a coderef between interpreters:

perl -Mthreads -Mthreads::shared -Mstrict 
-wle"
    my $c:shared; 
    async{ 
        ## This next line triggers the "Invalid value for shared scala
+r"
        $c = sub{ print 'Hi'}; 
        sleep 100
    }->detach;; 
    sleep 3;
    $c->();
    sleep 100"

Thread 1 terminated abnormally: Invalid value for shared scalar at -e 
+line 1.
Can't use an undefined value as a subroutine reference at -e line 1.

[download]

But, by passing the coderef via C, you are by-passing this protection. Hence, as I surmised, if the callback gets triggered at a point in teh application when the "wrong" interpreter has the cpu, then it fails. Whereas if the originating inpterpreter is in control, it works,

The solution I'm afraid is to re-think your methodology. If you can describe why you are starting a second interpreter, we might be able to see a solution.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
RIP an inspiration; A true Folk's Guy

Replies are listed 'Best First'.
Re^4: Perl crash during perl_clone
by perlmonk1729 (Acolyte) on Oct 28, 2010 at 06:01 UTC
    if the callback gets triggered at a point in teh application when the "wrong" interpreter has the cpu, then it fails

    There are only two threads/interpreters and the thread that cloned an interpter does nothing else but wait for the right event to invoke the callback. So, bit confused what could be the 'wrong' interprter

    The solution I'm afraid is to re-think your methodology. If you can describe why you are starting a second interpreter, we might be able to see a solution.

    Infact, I did not know anything about interpters or cloning when I started this project. I learnt how to modify the SWIG generated wrapper to register the callback (RegisterCB in my example): I cache the perl sub reference and instead register a proxy C method (InvokeCB) into the C-api interface. When an unrelated thread (BGThread) called InvokeCB, it sets up the call-stack properly to call perl subroutines and then calls the sub via the cached reference.

    However, the "call_sv" would always crashed - 100%. Then I stumbled across the perl_clone() and found many posts that claimed it was required because BGThread cannot call code into an interpreter it did not create (i.e perl.exe).

    Having a cloned interpter allowed call_sv to work reliably..until my latest problems. You can see this link about my previous experiences that got me here. http://markmail.org/thread/cjbhybjfikuirvud

    ps: I'm in asian timezone, so responses may seem to have a 'lag'.
[reply]
      There are only two threads/interpreters and the thread that cloned an interpter does nothing else but wait for the right event to invoke the callback. So, bit confused what could be the 'wrong' interprter ... because BGThread cannot call code into an interpreter it did not create (i.e perl.exe).

      We agree that there are two interpreters. We agree that it is important that the "right" interpreter be used for the callback.

      The problem is, if the callback is triggered by an asynchronous event, how can you guarantee that only right interpreter is running when the callback occurs?

      Because if you don't take steps to ensure that, then it will inevitably happen that sometimes the triggering event will occur during the time-slice of the wrong interpreter, and it will segfault.

      Have you tried my suggestion of logging the current thread id from a) the place where you get the Perl callback address; b) where you invoke that Perl callback address?

      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
        I expected the thread-ids to be different by virtue of my design. Anycase, I logged the 'current' thread-id and confirmed that the id when I get the perl-callback address (in the SWIG wrapper) is different (it must be a thread created by perl.exe) from when the perl cback is invoked (the thread I created). No surprise.

        However, I got thinking about what you mean by the 'right' interpreter..if I have two interpreters then it is not clear what is the 'right' interpreter. Can you elaborate which you think is the right interpreter?

        Then I decided to question the premise/info under which I went to the cloning method and did an experiment: I removed the perl_clone completely (and the related SvSHARE, sv_dup etc). Instead, when my "BGThread-equivalent" is created, I just forced a PERL_SET_CONTEXT() to the interpreter created by perl.exe

        This forces the callback thread to invoke the perl sub in the same interpreter context as that created by perl.exe itself.

        In my limited tests, very surprisngly for me, so far that seems to work as well as the perl_clone approach I used for almost a yr now.

        I'll be testing more, but does this surprise you?

[reply]

In Section Seekers of Perl Wisdom