Freeing memory, revisited (Linux)

lkundrak has asked for the wisdom of the Perl Monks concerning the following question:

Dear monks, A lot has been written about Perl memory management, but I could not find anything that would document the behavior I'm seeing. Using native malloc and allocating large chunks (more than 128k, so that mmap is used, not brk), I'm seeing the chunks being unallocated a bit too late. One example worth thousand double-words:

$ strace -e mmap2,munmap perl -e '{"x" x 1048576}{"x" x 1048576}{"x" x
+ 1048576}' 2>&1 |tail -6
mmap2(NULL, 1052672, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, 
+-1, 0) = 0xb740c000
mmap2(NULL, 1052672, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, 
+-1, 0) = 0xb730b000
mmap2(NULL, 1052672, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, 
+-1, 0) = 0xb720a000
munmap(0xb720a000, 1052672)             = 0
munmap(0xb730b000, 1052672)             = 0
munmap(0xb740c000, 1052672)             = 0
[download]

Anyone has an explanation for that, or pointer to documentation before I dig into the code?

Comment on Freeing memory, revisited (Linux) Download Code

Replies are listed 'Best First'.
Re: Freeing memory, revisited (Linux) by Anonymous Monk on May 05, 2011 at 11:16 UTC
Anyone has an explanation for that, or pointer to documentation before I dig into the code? The most common thing I've seen written is that 1) you shouldn't worry about it 2) perl will free memory when its good and ready	[reply]
Re: Freeing memory, revisited (Linux) by dave_the_m (Monsignor) on May 05, 2011 at 12:55 UTC
The three x expressions are constant folded at compile time, then the results are stored as part of the program code, and are only freed when the program is freed. Dave.	[reply] [d/l]
Re^2: Freeing memory, revisited (Linux) by Eliya (Vicar) on May 05, 2011 at 14:31 UTC
The three x expressions are constant folded at compile time Nice theory... but if the size to allocate isn't known before run-time, you still do get the same behavior. Even in slightly modified cases like these, where the `$y` should definitely go out of scope (and thus the memory behind it be available for reuse): $ strace -e mmap,munmap perl -e '$n=220; $x="x" x $n; {my $y=$x}{my +$y=$x}{my $y=$x}' ... mmap(NULL, 1052672, PROT_READ\|PROT_WRITE, MAP_PRIVATE\|MAP_ANONYMOUS, - +1, 0) = 0x7fa5d43d3000 mmap(NULL, 1052672, PROT_READ\|PROT_WRITE, MAP_PRIVATE\|MAP_ANONYMOUS, - +1, 0) = 0x7fa5d2ecd000 mmap(NULL, 1052672, PROT_READ\|PROT_WRITE, MAP_PRIVATE\|MAP_ANONYMOUS, - +1, 0) = 0x7fa5d2dcc000 mmap(NULL, 1052672, PROT_READ\|PROT_WRITE, MAP_PRIVATE\|MAP_ANONYMOUS, - +1, 0) = 0x7fa5d2ccb000 mmap(NULL, 1052672, PROT_READ\|PROT_WRITE, MAP_PRIVATE\|MAP_ANONYMOUS, - +1, 0) = 0x7fa5d2bca000 munmap(0x7fa5d2bca000, 1052672) = 0 munmap(0x7fa5d2ccb000, 1052672) = 0 munmap(0x7fa5d2dcc000, 1052672) = 0 munmap(0x7fa5d43d3000, 1052672) = 0 [download] Personally, I'd consider this a bug. Note that if you write `for (1..3) {my $y = $x}` instead of spelling out `{my $y = $x}` three times, the memory is being reused just fine: `$ strace -e mmap,munmap perl -e '$n=220; $x="x" x $n; for (1..3) {my + $y=$x}' ... mmap(NULL, 1052672, PROT_READ\|PROT_WRITE, MAP_PRIVATE\|MAP_ANONYMOUS, - +1, 0) = 0x7f9d9bdfd000 mmap(NULL, 1052672, PROT_READ\|PROT_WRITE, MAP_PRIVATE\|MAP_ANONYMOUS, - +1, 0) = 0x7f9d9a8f7000 mmap(NULL, 1052672, PROT_READ\|PROT_WRITE, MAP_PRIVATE\|MAP_ANONYMOUS, - +1, 0) = 0x7f9d9a7f6000 munmap(0x7f9d9a7f6000, 1052672) = 0 munmap(0x7f9d9bdfd000, 1052672) = 0` [download]	[reply] [d/l] [select]
Re^3: Freeing memory, revisited (Linux) by dave_the_m (Monsignor) on May 05, 2011 at 15:09 UTC
Nice theory Simply the truth. In your run-time example, a different mechanism causes the allocations to be retained. Every sub-expression that returns a result has a "pad temp" allocated to it to store the temporary result. The values in the pad temp are retained until the sub is freed, but they are re-used. i.e. in the following: `sub f { return "x" x $_[0] } f(100_000); f(5); f(200_000);` [download] a single temp will be stored within f's pad. On the first call it will hold a scalar of length 100_000, which gets retained. On the second call, the scalar is shrunk to length 5 (but not freed), and on the third call the scalar is realloced to grow to 200_000, and finally freed when the sub f is destroyed. Dave.	[reply] [d/l]
Re^4: Freeing memory, revisited (Linux) by Eliya (Vicar) on May 05, 2011 at 15:47 UTC
Re^5: Freeing memory, revisited (Linux) by dave_the_m (Monsignor) on May 05, 2011 at 16:44 UTC
Re: Freeing memory, revisited (Linux) by anonymized user 468275 (Curate) on May 05, 2011 at 11:44 UTC
Too late for what? This example looks like a Perl proof of Schroedinger's cat. When you look at the memory it stays alive - it gets garbage-collected only when you are not looking ;) One world, one people	[reply]