in reply to Return array or list from an XSUB?

You can't return arrays from subroutines. You can only return (a list of) scalars.

You can return an array ref by specifying an AV* return type - see this section of perlxs (take note of the remark about refcounts).

Returning lists is a bit more involved, lookup XPUSH in perlxstut and perlapi

So in short, returning an array ref is easier to code and can be significantly more efficient if you're returning a lot of values, but it can also be "uglier" to use, especially if you're returning multiple values that really are mostly used as different things instead of a big list.

Returning a list is more work to code, but may lead to a nicer API for the perl side.

"What should it profit a man, if he should win a flame war, yet lose his cool?"

Replies are listed 'Best First'.
Re^2: Return array or list from an XSUB?
by syphilis (Archbishop) on Nov 14, 2007 at 13:41 UTC
    So in short, returning an array ref is easier to code and can be significantly more efficient if you're returning a lot of values

    Been playing around with some stuff, trying to verify the above quote.

    I came up with this (perhaps it's sophistic): 
    use warnings;
use Benchmark;

use Inline C => Config =>
    BUILD_NOISY => 1;

use Inline C => <<'EOC';

void foo(AV * x) {
     dXSARGS;
     SV ** elem;
     int len, i;
     
     len = av_len(x) + 1;

     sp = mark;

     for(i = 0; i < len; i++) {
        elem = av_fetch(x, i, 0);
        XPUSHs(*elem);
     }

     PUTBACK;
     XSRETURN(len);
}

AV * bar(AV * x) {
     dXSARGS;
     SV ** elem;
     int len, i;

     sp = mark;

     len = av_len(x) + 1;

     for(i = 0; i < len; i++) {
        elem = av_fetch(x, i, 0);
        XPUSHs(*elem);
     }

     return x;
}

EOC

$len = 2000000;

@x = ();

for(1 .. $len) {push @x, $_}

timethese (1, {
 'ref'  => '$ret = bar(\@x)',
 'list' => '@ret = foo(\@x)',
});

    [download]
    For me that produces: 
    Benchmark: timing 1 iterations of list, ref...
      list:  1 wallclock secs ( 0.36 usr +  0.11 sys =  0.47 CPU) @  2
+.13/s (n=1
)
            (warning: too few iterations for a reliable count)
       ref:  0 wallclock secs ( 0.11 usr +  0.00 sys =  0.11 CPU) @  9
+.17/s (n=1
)
            (warning: too few iterations for a reliable count)

    [download]
    The difference is certainly *measurable*, but I wonder if it's all that *significant* (given the size of the array). I also wonder a little about the assertion that it's easier to return an AV* than a list. For most of the stuff that I have done, I've found it easier to return a list - even though the coding may involve additional keystrokes. (Admittedly, most of the stuff I've done is Inline based ... and I'm a simpleton :-)

    Cheers,
    Rob
[reply]
[d/l]
[select]
      The difference is certainly *measurable*....

      Not really. That's what "too few iterations" means. It's barely even significant either, as the difference between those two pieces of code is the addition of two stack-manipulating macros in the former case. You're doing about the same amount of work in both, so my guess is that there's another factor in your benchmark that more testing would expose.

[reply]
Re^2: Return array or list from an XSUB?
by smee30 (Initiate) on Nov 14, 2007 at 12:32 UTC
    Thanks Joost. Sorry, I should have said that I was returning an array ref from the XSUB, but then post-processing that in Perl. Like so: 
        $results = $self->_Run( @_ ); # This is the XSUB
    return @$results       if( wantarray );
    return undef           if( scalar( @$results ) == 0 );
    return $$results[ 0 ]  if( scalar( @$results ) == 1 );
    return $results;

    [download]
    That's the current implementation, but it needs to change as people don't seem to like the fact that it's context aware. They'd rather its return type was constant.
    Option 1 is to recode it like this: 
        $results = $self->_Run( @_ ); # This is the XSUB
    return @$results;

    [download]
    and simply have the perl wrapper always return an array. The other option is to use XPUSH to return a list instead.

    I'm trying to figure out what my users will gain from that implementation over the one above. Can you see anything?
[reply]
[d/l]
[select]
      Just a couple of thoughts.
      • It doesn't really matter if you implement this in XS or in a perl "wrapper" - use whatever works, both can do the same thing.

      • All perl statements that can return more than one value are context aware, since all statements can be called in scalar context and there is no fixed way of dealing with that. For instance:

        return (3, 2, 1)

        evaluates to 1 (the last element) in scalar context.

        @array = (3, 2, 1); return @array

        evaluates to 3 (the number of elements) in scalar context.

      It's just a matter of choosing whatever makes the most sense.

      If you're returning something that is "really" a long list of items, then returning the number of elements in scalar context (like an array) is a good choice.

      If you're returning something that is more like a "main" value plus "additional information" then it may make more sense to return the main value in scalar context.

      If you really want to return the same thing in both context, you should just return an array ref.

      "What should it profit a man, if he should win a flame war, yet lose his cool?"
[reply]
[d/l]
[select]
        Great - thanks!
[reply]

In Section Seekers of Perl Wisdom