perl5ever has asked for the wisdom of the Perl Monks concerning the following question:

With perl 5.12 are there any efficiency concerns to be aware of when returning a large arrays from a function?

E.g., does perl avoid copying of data in the following cases?

sub large_array { my @large_array; ...populate @large_array... return @large_array; } sub large_array_ref { my $large_array_ref = []; ...populate @$large_array_ref... return $large_array_ref; } # Is the returned array copied? @c = large_array(); @c = @{ large_array_ref() };

Replies are listed 'Best First'.
Re: returning large arrays from functions
by BrowserUk (Patriarch) on Nov 18, 2010 at 19:23 UTC
    E.g., does perl avoid copying of data in the following cases?

    No. The array is copied in both those cases.

    There are two ways of avoiding that:

    1. Return an arrayref as in your second example and use it as an array ref in the calling code:
      sub large_array_ref { my $large_array_ref = []; ...populate @$large_array_ref... return $large_array_ref; } my $c = large_array_ref(); $c->[ ... ];
    2. Pass an array ref into your sub:
      sub large_array_ref { my $large_array_ref = shift; ...populate @$large_array_ref... return; } large_array_ref( \my @c ); $c[ ... ];

      With a suitably chosen name for the sub, the latter looks less awkward:

      populateThingArray( \my @c ); $c[ ... ];

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      > With a suitably chosen name for the sub, the latter looks less awkward:

      populateThingArray( \my @c ); $c[ ... ];

      as a side note, one can use the fat comma for syntactic sugar:

      calcArrays( $par1, $par2 => \ my (@a,@b) );

      Cheers Rolf

      Update: extended code to return multiple Arrays.

Re: returning large arrays from functions
by ikegami (Patriarch) on Nov 18, 2010 at 20:15 UTC

    Perl does not employ a copy-on-write mechanism. As such, assigning a one array to another (as in the second example) involves copying the elements, not just copying a reference to the elements.

    In the first example, you're not even assigning an array to an array. You're getting a list of the elements in the array, then you're assigning that list to another array. You might end up copying each element twice! I know there's at least one optimisation in place, but there's no escaping copying each element at least once for the reason I gave above.

      Did the OP, or anyone, ever, suggest that Perl might use a "copy-on-write mechanism"?

        You seem to have misunderstood what I said. Whether copying occurs in an array to array assignment or not depends on whether Perl does COW or not. If it does, copying can be avoided. If it doesn't copying must occur. It doesn't, so copying must occur. And thus, the OP's question is answered.
Re: returning large arrays from functions
by locked_user sundialsvc4 (Abbot) on Nov 18, 2010 at 21:03 UTC

    Pass around a reference to the result array.   A reference (to anything...) occupies only one small slot on the runtime stack.

    Bear in mind that changes to any reference to “the same thing” will be reflected in all references to that “thing.”   If you need to make a deep copy of whatever the reference points to, there are several ways to do that, e.g. dclone() in Storable.

Re: returning large arrays from functions
by CountZero (Bishop) on Nov 18, 2010 at 22:51 UTC
    Have a look at Data::COW: clone deep data structures copy-on-write.

    CountZero

    A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James