in reply to Re: out of memory problem
in thread out of memory problem

Note that because you don't use "my" here, all these variables will stay in memory even after the loop has finished.

That happens when using my too. Neither the SV nor the string buffer are freed. They are simply cleared.

Update: The following isn't proof, but it does let you see the effect of what I described. 

use Devel::Peek qw( Dump );

sub foo {
   my ($x) = @_;
   undef($x) if !@_;
   Dump($x);
}


foo("abc");
foo("defghijkl");
foo("mno");
foo(undef);  # $x = undef
foo();       # undef $x
foo("pqr");

[download]

Pay particular attention to LEN, the allocated length of the string buffer associated with the variable. 

SV = PV(0x22612c) at 0x22606c
  REFCNT = 1
  FLAGS = (PADBUSY,PADMY,POK,pPOK)
  PV = 0x22f4d4 "abc"\0
  CUR = 3
  LEN = 4
SV = PV(0x22612c) at 0x22606c       Could be coincidence, but isn't.
  REFCNT = 1                          The SV wasn't freed.
  FLAGS = (PADBUSY,PADMY,POK,pPOK)
  PV = 0x182cadc "defghijkl"\0      New larger (LEN=12) str buf.
  CUR = 9
  LEN = 12
SV = PV(0x22612c) at 0x22606c
  REFCNT = 1
  FLAGS = (PADBUSY,PADMY,POK,pPOK)
  PV = 0x182cadc "mno"\0            Same "large" (LEN=12) buf reused.
  CUR = 3
  LEN = 12
SV = PV(0x22612c) at 0x22606c
  REFCNT = 1
  FLAGS = (PADBUSY,PADMY)           Undefined
  PV = 0x182cadc "mno"\0            Str buf untouched.
  CUR = 3
  LEN = 12
SV = PV(0x22612c) at 0x22606c
  REFCNT = 1
  FLAGS = (PADBUSY,PADMY)           Undefined
  PV = 0                            Str buf freed.
SV = PV(0x22612c) at 0x22606c
  REFCNT = 1
  FLAGS = (PADBUSY,PADMY,POK,pPOK)
  PV = 0x22f4d4 "pqr"\0             New small (LEN=4) str buf.
  CUR = 3                             Same address as earlier
  LEN = 4                             is a coincidence.

[download]

Replies are listed 'Best First'.
Re^3: out of memory problem
by betterworld (Curate) on Sep 13, 2008 at 14:16 UTC

    You're right. The reuse of the address could be coincidence, but as the LEN stay 12, we can conclude that the buffer was never freed.

    I've done a little bit of testing too:

    Read more... (2 kB)

    One question remains: When does perl free memory? I can't quite believe that every lexical variable that has ever been used results in large stale memory blocks. This would be against the spirit of this excerpt from perlsyn:

    You wouldn't want memory being free until you were done using it, or kept around once you were done. Automatic garbage collection takes care of this for you.
[reply]
[d/l]
[select]

      but it does allocate new memory before running bar, even though it could reuse $var's memory for $var2.

      No, that would defy the optimization of not deallocating it in the first place.

      For Perl to know when $var is not currently being used, it would have to 1) place $var on a free list when it stops being used and 2) remove it from the free list when $var is in use again. Those two operations are known as deallocation and allocation, exactly what the optimization is trying to avoid!

      Keep in mind that each variable consists of two memory blocks, three if they have a string buffer associated with them. And that's just for non-magical scalars. A lot of CPU time is being saved.

      One question remains: When does perl free memory?

      Anon SVs are freed. SVs that become anon (such as return values) are freed. SVs passed to undef have their attached buffer freed.

      Keep in mind that all of this is the result of an (undocumented?) optimization. It shouldn't be relied upon. It's just that you can't rely on it not happening.

[reply]
[d/l]
[select]

        Suppose I write a module which provides a "frobnicate" function:

        sub frobnicate {
    my ($string) = @_;
    $string =~ s/foo/bar/g;
    return $string;
}

        [download]

        Suppose I write a program that runs for a really long time, and uses "frobnicate" only once during its initialization, and it passes a really long string to it.

        Should I expect that $string will hold this long string for the program's lifetime? Am I supposed to undef every single variable to avoid this? (Which would pretty much nullify this "optimization", but it seems to be the only way to avoid the memory leak.)

        Update: Replaced the code of the example function.

[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom