Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Does undef free ram?

by Xxaxx (Monk)
on Jul 17, 2002 at 05:53 UTC ( [id://182343]=perlquestion: print w/replies, xml ) Need Help??

Xxaxx has asked for the wisdom of the Perl Monks concerning the following question:

Until now all of my Perl scripts have delt with smallish arrays and smallish hashes. Now I have need to run a script which will work on a few largish arrays.
I do not need all of the arrays at once, hence I can undef an array after I am done with it then move on to the next array.

I'm wondering if the undef is the best way to "throw out" the old array.

And I'm wondering if this will free the memory for subsequent use.

Sorry if the question is a bit odd but I have used languages in the past which did not handle freeing of memory properly and I'd rather not find out by hitting a brick wall when I scale later.

Thanks in advance.
Claude

Replies are listed 'Best First'.
Re: Does undef free ram?
by PetaMem (Priest) on Jul 17, 2002 at 07:53 UTC
    Hi,

    you have to distinguish between "free memory" that is available to the Perl process and "free memory" that is available to the operating system. A simple undef will not free the memory for the OS, but that for perl only.

    Let's say your box has 128MB RAM. You start some perl script . The perl process takes something between 5 to 6MB of RAM. Now you create some hash of 10MB size. Your process now takes 16MB of RAM. Now you undef the hash again. Your process still takes 16MB of ram.

    Now you - in the same process - create some 5MB list/array/whatever. Your process still takes 16MB of RAM. So perl keeps always the memory that was allocated at max.

    Subsequent allocations within this memory are handled by perl itself and not by the OS, thus the process doesn't grow in memory as long as you don't allocate more than was allocated at max. at any time before by the current perl process.

    So what can one do against that? Under Linux or some other superior OS that has fork - yes you got it: dealocate the big array and just fork. More precisely spawn and let the parent process die.

    Bye
     PetaMem

      Forking after using the array (regardless whether you undef it first) only makes the problem worse, now you have doubled the memory usage. Letting the parent die just makes you back at the previous memory usage.

      There is a "famous" trick, used by long running processes to prevent possible memory leakage from sucking up all the resources, but that doesn't use fork, it uses exec. execing yourself means that you are restarting yourself - starting with a fresh sheet of memory. Obviously, this is not going to work for many programs.

      Abigail

        What you say is true. However for many purposes it also works to fork before doing whatever will take memory. Then the bloated child who used memory can exit, leaving the parent lean and trim.

        Doing work within system calls can help for the same reason.

Re: Does undef free ram?
by zejames (Hermit) on Jul 17, 2002 at 08:12 UTC
    Here is a little script for linux only that can help to understand what happens to memory:

    our @test; print "nothing : Size = ".size()."\n"; push @test, rand for (1..100000); print "push : Size = ".size()."\n"; #undef @test; @test = (); print "deleted : Size = ".size()."\n"; sub size { open DATA, "< /proc/$$/status" or die "Unable to read data : $!\n"; local $/; <DATA> =~ m/^VmSize:\s+(.*?)$/m; return $1; }

    If I use @test = ();, output is
    nothing : Size = 2988 kB push : Size = 5492 kB deleted : Size = 5492 kB

    and with undef @test;
    nothing : Size = 2988 kB push : Size = 5488 kB deleted : Size = 4976 kB

    Of course, this example is OS dependant. But you can notice that the first solution does not free any OS memory at all. As FatVamp said, Perl can then reuse the memory. It can be noticed that a few more memory is freed when using undef, but not all that was used.

    HTH

    --
    zejames
      and here's the same script under Win32 (NT 4.0/2000/XP):
      use Win32::API 0.20; my $GCP = new Win32::API( "kernel32", "GetCurrentProcess", "", "N" ) or die "Can't import API: $^E"; my $GPMI = new Win32::API( "psapi", "GetProcessMemoryInfo", "NPN", "N" ) or die "Can't import API: $^E"; our @test; print "nothing : Size = ".size()."\n"; push @test, rand for (1..100000); print "push : Size = ".size()."\n"; #undef @test; @test = (); print "deleted : Size = ".size()."\n"; sub size { my $handle = $GCP->Call(); my $m = pack("LLLLLLLLLL", 16*10, 0 x 10); $r = $GPMI->Call( $handle, $m, 160 ); my $pwss = (unpack("LLLLLLLLLL", $m))[2]; return int($pwss/1024)." kB"; }
      interestingly enough, the figures are different here: if I use @test = ();, output is:
      nothing : Size = 2252 kB push : Size = 4776 kB deleted : Size = 4776 kB
      and with undef @test;
      nothing : Size = 2252 kB push : Size = 4776 kB deleted : Size = 4776 kB
      both solutions don't free any memory at all.

      cheers,
      Aldo

      __END__ $_=q,just perl,,s, , another ,,s,$, hacker,,print;
Re: Does undef free ram?
by bronto (Priest) on Jul 17, 2002 at 12:05 UTC

    When I need to allocate variables for a limited time, for example when I need them in just a snippet of code, I usually prefer blocks and lexicals instead of undefing. For example:

    { my @needed_here = get_array() ; # do something with data, e.g. coffee(@needed_here) ; } # you are no more @needed_here ;-)

    As soon as you get out of the block, @needed_here "disappears" (unless you saved some reference to it, for example in a closure)

    I prefer this way instead of undefing because it looks clearer to me --when I need the variable, it's there, when I need it no more it goes away (instead of staying around undefined).

    If you are interested in this approach, you could take a look at "Private Variables via my()" in perlsub

    Ciao!
    --bronto

    # Another Perl edition of a song:
    # The End, by The Beatles
    END {
      $you->take($love) eq $you->made($love) ;
    }

Re: Does undef free ram?
by Elian (Parson) on Jul 17, 2002 at 15:20 UTC
    Yes, undef is the best way to completely release the memory that an array uses. That memory will be available for reallocation in your program. The memory will not, unless the array is very large and you're using an OS or version of libc that supports it, be released back to system.

    In this case, where your array is 10K elements or so, it's probably not worth it. That's only using 40K of memory, which really isn't that much.

Re: Does undef free ram?
by Marza (Vicar) on Jul 17, 2002 at 06:11 UTC

    I think undef will be fine if you are dumping the hash or array.

    Also how many elements are you talking about? I have to ask because one of my partners was concerned about the same thing and he had 100 elements? ;)

    Many machines these days have tons of ram and it quickly recovers stuff once a job completes. So it is good to be clean but don't get too freeked out unless of course you have gigantic arrays and hashes.

    Just my .02

      I believe my arrays will be limited to 10,000 or less. Normally I wouldn't be concerned because I've never seen a problem with Perl on a Linux box holding ram after the job completes. I don't know if this is to the credit of Linux or Perl or both. But so far once the job completes it looks like all ram is returned.

      However since I'll be using a couple of dozen 10,000+ arrays one after the other during the same job I was concerned about memory during the job.

      Claude

Re: Does undef free ram?
by talexb (Chancellor) on Jul 17, 2002 at 13:10 UTC
    Just to echo what bronto already said -- let the arrays you don't need go out of scope and let Perl take care of the memory management. That's cleaner than using undef on the arrays.

    --t. alex

    "Mud, mud, glorious mud. Nothing quite like it for cooling the blood!"
    --Michael Flanders and Donald Swann

      Except this isn't entirely true--perl doesn't release the memory used for the array itself, though the memory for the scalars is released. If you want an array's memory released, you do have to undef it.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://182343]
Approved by Chady
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others studying the Monastery: (5)
As of 2024-03-29 10:50 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found