Re: Perl Optimization

I will be very interested in the outcome of such a study. We are in the process right at this moment profiling some data loading code that takes about 1/2 hour to run on some very beefy hardware (Sun Sparc 4500, 2 gig memory, etc...).

One of the things we are looking at is de-referencing. Assume you have a hash reference that is a complex data structure:

my $href = {
  'x' => {
     'y' => {
         'z' => [ $xref, $yref, $zref ],
      },
     'strange' => { ..... },
  },
  'charm' => { .... },
};
[download]

Now imagine that the example above is actually a reference to an enormous object, with grand-children, great-grand-children, and great-great-great-great-grandchildren. From what I can see, every time you de-reference some or all of the object, a copy is made.

foreach  my $obj  (%{$href})
# This dereference ^ causes a deep copy to be made
[download]

In the case where these are really big objects, as some of ours are, that would be rather rough, especially if we iterate through children, then grand-children, then deeper still.

At this point, the savings gained by passing a reference to a function is lost due to the fact that a copy of the referent is made in the function. For instance:

01: my $list = [1, 2, 3];
02: doStuff($list);
03: 
04: sub doStuff
05: {
06:   my $listRef = shift;
07:   foreach $listElement (@{$listRef})
08:   {
09:     print "$listElement \n";
10:   }
11: }
[download]

In line 7, a deep copy of the list is made. If we change the sub to take not a reference, but the whole list, is this equivalent, or is it different?
Of course, if in the subroutine you de-reference the thing more than once, you could be incurring huge overhead. This is not at all an issue with a puny example like the above, but we have rather hierarchical data, with objects that contain child objects, with children, etc. All method calls are by reference, and most references are de-referenced all over the place.

I can't send you our code (proprietary, you know) but I might be able to put together a better example to show you what I mean. Right now, one of our guys is looking in to this. I will publish the results of our study here.

Brian - a.k.a. DrSax

Comment on Re: Perl Optimization Select or Download Code

Replies are listed 'Best First'.
Re (tilly) 2: Perl Optimization by tilly (Archbishop) on Jun 01, 2001 at 18:37 UTC
Copies are not being made where you think they are.	[reply]
Re: Re: Perl Optimization by Cybercosis (Monk) on Jun 01, 2001 at 20:29 UTC
foreach my $obj (%{$href}) # This dereference ^ causes a deep copy to be made actually, this is more accurate: `foreach my $obj (%{$href}) #here ^ is the copy` [download] If you check the ref()-iness of $obj, you'll find that there isn't any at all. The appropriate value is copied into $obj on each run of the loop. I'm not at all sure what it is you're trying to solve with this, but if you need access to the key/value pair without the big, horrendous copy, maybe something like this: `foreach my $key (keys %{$href}) { #do what you need with $key and $href->{$key} }` [download] Hope you find this useful! ~Cybercosis nemo accipere quod non merere	[reply] [d/l] [select]
(tye)Re: Perl Optimization by tye (Sage) on Jun 01, 2001 at 21:29 UTC
for and foreach make the argument an alias to (not a copy of) the values in the list. Even if you copy a reference, you still get a reference, so I don't understand what you are trying to say. `my $href = { 'x' => { 'y' => { 'z' => [ $xref, $yref, $zref ], }, #'strange' => { ..... }, }, #'charm' => { .... }, }; for my $obj ( %$href ) { print "$obj: ",ref($obj),$/; my $copy= $obj; print "$copy: ",ref($copy),$/; } __END__ prints: x: x: HASH(0x1a65030): HASH HASH(0x1a65030): HASH` [download] - tye (but my friends call me "Tye")	[reply] [d/l]