in reply to Re: Reference to sort results
in thread Reference to sort results

Why bother? ...Because in reality I have an object containing a large hash, and I want one of its methods to return a sorted list of its values. More like: 
@sorted = sort { $a->{VAL} <=> $b->{VAL} } keys %bigHash;
return \@sorted;

[download]
Surely it is more efficient to return a reference to that list, as returning the array itself will involve creating a new copy of that array. If I use: 
return [ sort { $a->{VAL} <=> $b->{VAL} } keys %bigHash ];

[download]
Will that duplicate the array, or just return a reference to the extant array? It isn't clear what perl does internally with this stuff. I'm just trying to be efficient, rather than throwing around big arrays willy-nilly.

Replies are listed 'Best First'.
Re^3: Reference to sort results
by ssandv (Hermit) on Aug 05, 2009 at 15:22 UTC
    It's a reference to an anonymous array which is created in the same step as the return statement. Why would it duplicate it?

    Going from right to left: 
    
keys %bigHash

    [download]
    generates a list, containing the values in the hash. 
    
sort {...}

    [download]
    Manipulates the list according to the sort rule (sorts in ascending numerical order of the value associated with the key). Still a list. 
    [...] #now the brackets

    [download]
    the brackets tell Perl that "hey, this list in here? gimme a ref to an array with those elements in that order. This is the first point where it's an array in your example. It's unclear where you think copying an array is occurring. There's no practical difference between 
    my @foo=sort @bar;
return \@foo;

    [download]
    and 
    my $foo=[sort @bar];
return $foo;

    [download]
    unless you try to manipulate the array in the subroutine, in which case you'd need to deal with @$foo in the second example. sort operates on a list and returns a list. If you use an array to feed it, it doesn't change the original array. The array you're worried about copying isn't an array yet at that point, so there's no risk of copying it.

    (And as a general note, if you aren't observing what appears to be a resource-driven slowdown, you should usually not try to out-think the language implementation, because in certain circumstances you can interfere with compiler-level optimization)

    Edit: I think perlreftut might provide some enlightenment.
[reply]
[d/l]
[select]

      Thanks ssandv, that is really interesting. Could you tell me a little more?

      If I had an array, and wanted a reference to that exact same data, I would do this... 

      $aref = \@array;

      [download]

      Whereas, if I do this... 

      $aref = [ @array ];

      [download]

      ...Perl copies the contents of @array into a new anonymous array, then makes $aref a reference to the new one, right? So in this case @array and @$aref are entirely separate arrays.

      This is why I think: 

      $aref = [ sort ... ];

      [download]

      Would produce an array as the result of sort, copy that result into the new array implied by the brackets, give me a reference to the second array, and abandon the first one. It is not obvious whether Perl is smart enough to know that would be a waste of time.

      Furthermore, you say sort returns a list, not an array. As far as I am aware, Perl makes no explicit difference between the two, because its array data type can be used as an ordered list, so what do you mean by this?

      Thanks again for the help. I have been trawling the web but can't find anything which is clear on these points. I guess most people just don't worry about it!

[reply]
[d/l]
[select]
        Data: Arrays has a wealth of information about the difference between lists and arrays.

        It might help to note that @arr2=@arr1 *also* makes a copy of the array. The part you seem to be having trouble with is that arrays are data types and lists are temporary collections of scalars used as arguments. When you use an array as an argument where a list is expected, Perl treats the array elements as a list.
[reply]
[d/l]
        This sequence: 
        $aref = \@array;
$aref = [ @array ];

        [download]
        1. $aref becomes a pointer to the @array
        2. then $aref is reassigned as pointer to a "deep copy" of the @array in a new hunk of memory.

        "@array" is not "abandoned", that memory and the original @array can be accessed by the name @array.

        Perl uses reference counting when deciding to "free" memory. If you want to "free" @array, then @array=(); would do it provided that there aren't any other things like $aref2 = \@array hanging around out there. Maybe this point is too detailed here, but Perl never gives memory back to the OS. If you watch a long lived Perl process run, it will grow in size and approach a max size. It will never get smaller. So to be more specific, when Perl "frees" memory that means that memory can be recycled and reused by Perl itself, not the OS.

        This construct with square brackets $ref=[ 1, 2, 3] always allocates memory with "no name", an anonymous array.

        So,return[@something] returns a reference to a hunk of memory. If all you do with the return value is use the $ref (and don't make copies of that ref), then the second time you run the sub{}, then at the end of the day, that original anon memory will get "recycled", ie, this is not a memory leak.

        When I said "why bother?", that meant that putting this sort into a sub{} won't save any memory, nor will it increase efficiency. Just use the "one liner". I showed how to pass a ref to the a sub that sorts. If you want a sub that sorts, and intent is to modify the original array, this is the way to do it (no return value from the sub). I hope its clear that NO return value is faster than having a return value! This doesn't save any time either, actually slightly slower as the de-referencing takes some time. These Perl functions like sort{}, foreach(keys %hash) are very optimized. Try to use the idiomatic language constructs and some of them like the REGEX engine have become significantly faster between releases.

[reply]
[d/l]
[select]

        Thank you both for taking the time to respond.

        I think the ssandv hit the nail on the head: I did not know "list" meant anything other than the usual data structure in Perl. That perlfaq link makes it clear.

        Marshall's advice to trust Perl to do it right is probably also sound. Having worked in C/C++ for several years, I instinctively assume I have to do something clever to make my code efficient, and it is unnerving not to know what is going on under the hood. Perl coding needs a different mindset!

[reply]

In Section Seekers of Perl Wisdom