in reply to PerlXS typemap and reference counting

Are you saying that when you call new() from Perl code (something like):

use Grpc::XS::Call;
use Grpc::XS::Channel;
use Grpc::XS::Timeval;

my $timeval = Grpc::XS::Timeval->new( ... );
my $channel = Grpc::XS::Channel->new( ... );

my $call = Grpc::XS::Call->new( $channel, $timeval );
...

[download]

by the time the values arrive in your XS code, they are out of scope?

If so, then the likelihood is that they have already been GC'd before you call the constructor (easily checked) and the problem lies in your Timeval & Channel constructors.

You can manually increment the reference count using SvREFCNT_inc or its optimised variants; but be sure that you don't render them immortal and create a memory leak.

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
In the absence of evidence, opinion is indistinguishable from prejudice.

Replies are listed 'Best First'.
Re^2: PerlXS typemap and reference counting
by joyrex2001 (Initiate) on Apr 24, 2016 at 09:24 UTC
    > by the time the values arrive in your XS code, they are out of scope?

    No, at that point they are in scope. But the problem is, they are referenced within the object (and used in another XS method). Perl thinks they are out of scope after the constructor is called, but this isn't the case here. This means I need to increase the reference count and this is the part I don't know how to do in XS. How do I increase the reference count in the constructor, where I get the two objects (and decrease upon destroy).
    When writing this all down, I think I am better off refactoring parts of the XS code to perl and have the perl interpreter manage the reference counting...
[reply]
      I need to increase the reference count and this is the part I don't know how to do in XS

      As BrowserUK indicated, you can use SvREFCNT_inc/SvREFCNT_dec (see perldoc perlapi) to increment/decrement the reference count in XS, or you can write perl wrappers for those XS functions if you want to do the manipulating from perl space: 
      use strict;
use warnings;
use Devel::Peek;

use Inline C => Config =>
 CLEAN_AFTER_BUILD => 0,
 BUILD_NOISY => 1;

use Inline C => <<'EOC';


void incref(SV * sv) {
 SvREFCNT_inc(sv);
}

void decref (SV * sv) {
 SvREFCNT_dec(sv);
}

EOC

my $str = 'hello world';
Dump $str;
print "\n";

incref $str;
Dump $str;
print "\n";

decref $str;
Dump $str;

      [download]
      The REFCNT of those 3 dumps is 1, 2, and 1 respectively.

      I think I am better off refactoring parts of the XS code to perl and have the perl interpreter manage the reference counting

      That wouldn't be my first choice, but it's certainly an option - especially if it's proving difficult to handle the reference counting correctly.

      Cheers,
      Rob
[reply]
[d/l]
[select]

        FYI: Core module, Internals has GetRefCount() & SetRefCount(). {thinks for 10 seconds and adds...} but demoing I::C at this point is a very good idea ;)

        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
        In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
      No, at that point they are in scope. But the problem is, they are referenced within the object (and used in another XS method). Perl thinks they are out of scope after the constructor is called, but this isn't the case here.

      Okay. I get what you're saying now -- it would have been a whole lot clearer if you'd provided a full example; or were more precise in your description...

      They don't go out of scope in the constructor, not even immediately on return from it. They go out of scope when you reach the end of the block (subroutine or package) in which they were defined (my). Ie. they act exactly like normal perl lexical variables.

      You don't want them to act like normal perl variables, because you're storing them inside your XS structure, which perl doesn't know anything about, and so it duly GC's them once their Perl scope comes to an end.

      This means I need to increase the reference count and this is the part I don't know how to do in XS. How do I increase the reference count in the constructor, where I get the two objects (and decrease upon destroy).

      If my latest guess is correct -- you don't make it easy -- this is less about how to programmically increment and decrement the reference counts, but rather when to do so.

      And actually, you seem to have a pretty clear handle on that: You increment it when you first store it into your XS structure; and decrement it a) if you overwrite it with a different value; or b) when you are destroying the structure. (Ie. Add a DESTROY) method for the Call object.

      Maybe you already tried this and it didn't work? Gave errors?

      The problem is that you are using the T_PTROBJ typemap; and by the time your XS code gets control, the reference has already been unwrapped -- by the XS generated typemap code -- and what you get is not the SV_ref, but the value you put inside it. Which is useful, most times, but not if you need to manipulate the ref counts.

      The solution is to move from T_PTROBJ to T_OPAQUE, which means your XS code will be given the svref itself; which will allow you to manipulate the refcount; but also require you to unwrap the reference pointer yourself.

      If you posted a working -- or close to working -- version of your code; we could probably show you how to finish it.

      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
        Sharing the code in here is a bit hard (hence the github link in the first post, which points to the whole project). Without the gRPC library installed and without the complete project, you can't run it.

        Anyhow, you guessed right :-) Many thanks for that! I think the T_OPAQUE is exactly what I need. I can port to perl as well, but I think this has less impact and is fine as well.
[reply]
      Don't use malloc, use Newx , see perlclib
[reply]

In Section Seekers of Perl Wisdom