Nova has asked for the wisdom of the Perl Monks concerning the following question:

Hi All, I have never given much thought to memory deallocation with Perl because I have read in just about every book I own that Perl handles garbage collection automatically . I've written quite a bit of code with Perl over the past many years but did not really have to be too concerned about MEM usage . I am now working with larger arrays ( LoL's ) and wish to ensure deallocation of MEM but am a bit puzzled now because when I undef the @X array in code below , the system memory is not released . It is only released when the program exits. I'm using windows , with the Activestate perl binary install. 
{
my $i=0 , @X ;
while ( $i < 500000 ) {
push @X , ["a.".$i , "b.".$i , "c.".$i , "d.".$i ] ;
if ($i%10000==0) {printf "Index $i $X[$i][2] \n\n" ;}
$i++ ;
} ;
printf "Max X index Before undef %d\n",$#X ;
sleep 5 ;
undef @X ;
sleep 5 ;
printf "Max X index After undef %d\n",$#X ;
};
sleep 15 ;

[download]
Thank you for your insights . Nova

Replies are listed 'Best First'.
Re: memory deallocation
by oha (Friar) on Oct 01, 2007 at 14:25 UTC
    usually, the memory is freed by the gc to let be used again by the interpreter, but not freed to the OS. (if a malloc is called, the libc verify if there is enough space on data segment allocated by the process, if not the syscall sbrk is called to increase the data segment. i will be surprised to know that freeing enough memory could defragment it and release it to the OS, and i also suspect sbrk cannot be called to reduce it anyway)

    Oha

    update: after some tests, i found that allocating memory for an array will free to OS after undef-ing the array, but that will not apply to the data in the array itself: 

    my @x;
system "ps aux | grep $$ | grep [p]erl";
$x[1_000_000] = "foo";
system "ps aux | grep $$ | grep [p]erl";
undef @x;
system "ps aux | grep $$ | grep [p]erl";
push @x, "$_" for(1..1_000_000);
system "ps aux | grep $$ | grep [p]erl";
undef @x;
system "ps aux | grep $$ | grep [p]erl";
------
oha       8305  0.0  0.2  3296 1232 pts/0    S+   19:55   0:00 perl t4
+.pl
oha       8305  0.0  0.9  7204 5148 pts/0    S+   19:55   0:00 perl t4
+.pl
oha       8305  0.0  0.2  3296 1244 pts/0    S+   19:55   0:00 perl t4
+.pl
oha       8305 76.0  8.6 46896 44800 pts/0   S+   19:55   0:00 perl t4
+.pl
oha       8305 98.0  7.8 42796 40704 pts/0   S+   19:55   0:00 perl t4
+.pl

    [download]
    the mem freed to OS after the second undef is pretty close the same as the first (difference), so only the memory allocade by the strings aren't freed to OS. i still wonder why...
[reply]
[d/l]

      Actually some OSen do allow you to shrink your process size with sbrk (FreeBSD for one). I believe if your perl is using the system malloc you'll get this behavior there (FreeBSD). But in general you should only count on the behavior described above.

[reply]
        Windows is such a platform too. On Windows ActivePerl 5.8.8, the undef causes the memory usage to go from 171MB to 149MB. I wonder why so little memory is released.
[reply]
[d/l]
      Hi oha , I am very grateful to you and the other people who have shared their insights with me regarding the question I posed earlier today . I hope that those other than oha will also receive this thank you note , I am just learning how to get around at this time , within this amazing forum . In my initial code , I thought that because I was pushing anonymous arrays into @X , somehow , in the undef of @X the anonymous arrays still existed somewhere and took up MEM . With your code oha , you create and initialize a 1d array , and it seems as though undef behaves differently when the array elements have been initialized to some value / string as opposed to being unitialized. In my previous coding work , I had always assumed , undef -ing an array which say took up 100 MB , would release the 100 MB back to the system . I am still trying to assimilate the information you have all provided. Thank you . Nova .
[reply]
Re: memory deallocation
by kyle (Abbot) on Oct 01, 2007 at 14:45 UTC

    It looks as if undef does the job on my system (Linux ... 2.6.20-16-generic #2 SMP ... i686). 

    system "ps axl | head -1";
my @x; $x[1_000_000] = undef;
system "ps axl | grep $$";
undef @x;
system "ps axl | grep $$";

    [download]

    VSZ goes from 9040 before the undef to 5132 after.

[reply]
[d/l]
[select]
Re: memory deallocation
by cdarke (Prior) on Oct 01, 2007 at 15:13 UTC
    > I have read in just about every book I own that Perl handles garbage collection automatically

    Being pedantic, Perl 5 does not have a garbage collector, it uses reference counting.
    Few languages and run-time environments are capable of returning memory to the OS. Windows does allow so-called 'user-heaps', but I don't think many people use them. Even so Windows only allows whole unused pages (4kb/page on 32-bit Intel) to be returned to the OS, heap fragmentation is a common issue which will prevent this.
    This does not happen automatically, the application (in this case, perl) has to call the API, usually HeapCompact.
[reply]

      Perl 5 does not have a garbage collector, it uses reference counting.

      According to Wikipedia, "Reference counting is often known as a garbage collection algorithm." I, for one, have always considered reference counting to be a simple garbage collector.

[reply]
Re: memory deallocation
by shmem (Chancellor) on Oct 02, 2007 at 08:20 UTC 
    my $i=0 , @X ;

    [download]

    Nope. Consider: 

    my $i=0 , @X;
print "\@X is a global!\n" if defined $main::{X};
__END__
@X is a global!

    [download]

    You want: 

    my ($i, @X) = 0; # $i gets the zero, @X gets nothing

    [download]

    or better 

    my $i = 0;
my @X;

    [download]

    --shmem 

    _($_=" "x(1<<5)."?\n".q·/)Oo.  G°\        /
                              /\_¯/(q    /
----------------------------  \__(m.====·.(_("always off the crowd"))."·
");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}
[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom