Re^4: Memory management with long running scripts

Interesting. FWIW, on Windows 7 (monitoring with Task Manager) and running Strawberry 5.14.2.1, the program below will release (very) roughly half of the allocated memory when the undef $s is done. If the undef statement is changed to $s = ''; nothing is deallocated.

Half?!? What this means I do not know. (But see Update2 below.)

>perl -wMstrict -le
"my $s = '.' x 500_000_000;
 sleep 5;
 undef $s;
 sleep 10;
"
[download]

Update: Same results with Strawberry 5.10.1.5.

Update2: Actually, I think I understand a little of what is going on. From the Task Manager memory stats, when the '.' x 500_000_000 expression of the my $s = '.' x 500_000_000; statement executes, it consumes about 500M to build the string. The string is then copied to the $s scalar, thus consuming another 500M, for a total of 1G. It appears the first 500M (used to build the string) is never deallocated. Only the memory consumed by the $s scalar is deallocated when it is undef-ed.

Comment on Re^4: Memory management with long running scripts Select or Download Code

Replies are listed 'Best First'.
Re^5: Memory management with long running scripts by BrowserUk (Patriarch) on Jul 22, 2012 at 01:34 UTC
To create a large string in memory without the create-then-copy, use: `my $x = chr(0); $x x= 500e6;` [download] Now, when you undef $x, only the original 1 byte constant remains. All the rest is released back to the OS. With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. The start of some sanity?	[reply] [d/l]
Re^6: Memory management with long running scripts by AnomalousMonk (Archbishop) on Jul 22, 2012 at 01:45 UTC
Ah! Works like a charm.	[reply]
Re^5: Memory management with long running scripts by bulk88 (Priest) on Jul 22, 2012 at 00:36 UTC
Update2: Actually, I think I understand a little of what is going on. From the Task Manager memory stats, when the '.' x 500_000_000 expression of the my $s = '.' x 500_000_000; statement executes, it consumes about 500M to build the string. The string is then copied to the $s scalar, thus consuming another 500M. it appears the first 500M (used to build the string) is never deallocated. Only the memory consumed by the $s scalar is deallocated when it is undef-ed. You have a 500MB constant folded string sitting in RAM created by the Perl Compiler/Parser. It stays around in the function's PAD aslong as the function's opcodes stay around (usually entire runtime). Undef guarantees it will free the string buffer in the scalar. "$scalar = undef();" is not the same as "undef($scalar);", "= '';" just truncates SvCUR to 0. If you want "= ''" to free the string buffer, bring it in to Perl RT and up with p5p. I think perhaps over a certain cutoff (low MBs) assigning empty string or undef, should trigger a free, similar to what undef($scalar) does.	[reply]
Re^6: Memory management with long running scripts by AnomalousMonk (Archbishop) on Jul 22, 2012 at 01:33 UTC
Is there any way to persuade Perl to release its death-grip on the memory used by `'.' x 500_000_000` to build the string in the first place?	[reply] [d/l]