in reply to Re: Memory management with long running scripts
in thread Memory management with long running scripts

To my knowledge no. Perl does not "free" memory back to the OS, once it has it, it is not returned. There is a big difference in allowing Perl to reuse the memory that it already has for itself (e.g. "destroying Perl objects, etc).

Not true on Windows Perl, on unix perl, I dont know. I've heard on PerlMonks that unix malloc uses one continuous memory block for malloced memory that grows upwards sequentially (sbrk style). MS C Lib/Windows malloc uses different non contiguous pools and allocations over a certain size basically go straight to the VM paging system (mmap style) and get random blocks of paging memory. According to p5p, until this or last month, compiled OPs were not freeable or something similar. Weak references used to leak in 5.10, and I think it was fixed in 5.10.1 (personally ran into that). So there is a realistic chance your leak in Perl and not XS modules. 5.8 is very old.

Update, weak ref leak is https://rt.perl.org/rt3/Public/Bug/Display.html?id=56908.
  • Comment on Re^2: Memory management with long running scripts

Replies are listed 'Best First'.
Re^3: Memory management with long running scripts
by Marshall (Canon) on Jul 21, 2012 at 23:13 UTC
    I am certainly willing to learn something new!
    Can you make a Perl process on Windows that allocates and uses a large amount of physical memory, say 500 MB worth. And then show that Perl "released it" to the OS? Without the Perl process termination? I am running 5.10.1 Perl.
[reply]

      Interesting. FWIW, on Windows 7 (monitoring with Task Manager) and running Strawberry 5.14.2.1, the program below will release (very) roughly half of the allocated memory when the undef $s is done. If the  undef statement is changed to  $s = ''; nothing is deallocated.

      Half?!? What this means I do not know. (But see Update2 below.) 

      >perl -wMstrict -le
"my $s = '.' x 500_000_000;
 sleep 5;
 undef $s;
 sleep 10;
"

      [download]

      Update: Same results with Strawberry 5.10.1.5.

      Update2: Actually, I think I understand a little of what is going on. From the Task Manager memory stats, when the  '.' x 500_000_000 expression of the  my $s = '.' x 500_000_000; statement executes, it consumes about 500M to build the string. The string is then copied to the  $s scalar, thus consuming another 500M, for a total of 1G. It appears the first 500M (used to build the string) is never deallocated. Only the memory consumed by the  $s scalar is deallocated when it is undef-ed.

[reply]
[d/l]
[select]

        To create a large string in memory without the create-then-copy, use:

        my $x = chr(0); $x x= 500e6;

        [download]

        Now, when you undef $x, only the original 1 byte constant remains. All the rest is released back to the OS.

        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

        The start of some sanity?
[reply]
[d/l]
        Update2: Actually, I think I understand a little of what is going on. From the Task Manager memory stats, when the '.' x 500_000_000 expression of the my $s = '.' x 500_000_000; statement executes, it consumes about 500M to build the string. The string is then copied to the $s scalar, thus consuming another 500M. it appears the first 500M (used to build the string) is never deallocated. Only the memory consumed by the $s scalar is deallocated when it is undef-ed.

        You have a 500MB constant folded string sitting in RAM created by the Perl Compiler/Parser. It stays around in the function's PAD aslong as the function's opcodes stay around (usually entire runtime). Undef guarantees it will free the string buffer in the scalar. "$scalar = undef();" is not the same as "undef($scalar);", "= '';" just truncates SvCUR to 0. If you want "= ''" to free the string buffer, bring it in to Perl RT and up with p5p. I think perhaps over a certain cutoff (low MBs) assigning empty string or undef, should trigger a free, similar to what undef($scalar) does.
[reply]

In Section Seekers of Perl Wisdom