Shrunk array takes more memory than original

kroach has asked for the wisdom of the Perl Monks concerning the following question:

I'm using the $# feature to shrink an array. After shrinking to half the size, the array seems to take a bit more memory. How is that possible? (perl 5.10 on 64-bit GNU/Linux)

use strict;
use warnings;
use feature qw(say);
use Devel::Size qw(size);

my @array = 1 .. 10_000;
say size(\@array);
$#array = 5_000;
say size(\@array);
[download]

Output:

80064
80216
[download]

Is there a way to shrink the array without causing it to grow (and probably be reallocated)?

UPDATE: It seems splice doesn't cause the same effect.

my @array = 1 .. 10_000;
say size(\@array);
splice @array, 5_001;
say size(\@array);
[download]

Output:

80064
80064
[download]

Furthermore, after running a few simple benchmarks, it turns out the splice method is about as fast as last index assignment, so it seems splice doesn't do unnecessary copying in void context.

Comment on Shrunk array takes more memory than original Select or Download Code

Replies are listed 'Best First'.
Re: Shrunk array takes more memory than original by BrowserUk (Patriarch) on Nov 10, 2015 at 13:04 UTC
Using `total_size()` rather than `size()`: `@a = 1 .. 10000;; print total_size \@a;; 320176 $#a = 5000;; print total_size \@a;; 200376` [download] As you can see, the total size occupied by the array has shrunk by 30%. When you initialise the array, you cause perl to allocate 10,000 * 8 (64-bit) pointers as a continuous piece of memory for the array, plus 10,000 * 24 bytes for the scalars = 320,000 bytes (plus a little extra for the array mechanics. When you assign to $#array, the size of the base array does not get reallocated (hence the size() doesn't change), but half of the scalars are reclaimed. So now you have 10,000 * 8 plus 5,000 * 24 = 200,000 (plus the little extra). With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :) In the absence of evidence, opinion is indistinguishable from prejudice.	[reply] [d/l] [select]
Re: Shrunk array takes more memory than original by choroba (Cardinal) on Nov 10, 2015 at 12:26 UTC
The only way how to shrink the memory is to allocate a new array (or undef, but then you can't keep any values after shrinking). `my @shrunk = @array[0 .. 5_000]; # Array slice. say size(\@shrunk); # 40072` [download] See also How do I shrink an array in Perl? on StackOverflow. لսႽ† ᥲᥒ⚪⟊Ⴙᘓᖇ Ꮅᘓᖇ⎱ Ⴙᥲ𝇋ƙᘓᖇ	[reply] [d/l]
Re^2: Shrunk array takes more memory than original by kroach (Pilgrim) on Nov 10, 2015 at 12:45 UTC
I do not mean to shrink the memory, I would like it to stay the same instead of growing. I'm not adding anything new to the array and I don't get why it gets larger from having half of its elements removed.	[reply]
Re^3: Shrunk array takes more memory than original by BrowserUk (Patriarch) on Nov 10, 2015 at 13:06 UTC
why it gets larger from having half of its elements removed. Because using $#array causes some magic to be attached to the array. You don't even have to assign to $#array for this to happen, only reference it: `@b = 1 .. 10; print size \@b;; 256 print $#b; print size \@b;; 9 440` [download] And yes, 184 bytes does seem excessive. You'd think that the magic could be stored once and simply referenced for each array using an 4/8-byte pointer. With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :) In the absence of evidence, opinion is indistinguishable from prejudice.	[reply] [d/l]
Re^4: Shrunk array takes more memory than original by dave_the_m (Monsignor) on Nov 10, 2015 at 13:12 UTC
Re^5: Shrunk array takes more memory than original by BrowserUk (Patriarch) on Nov 10, 2015 at 13:16 UTC


P is for Practical
	PerlMonks