Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hi,
I'm trying to understand whether a large program I'm dealing with has memory leaks. I did some search, and in all the places where Perl's memory use is discussed, it says something like this:

- Perl's garbage collector works by counting references.
- If your code contains circular references, the garbage collector will not be able to free that memory, as the reference count won't go down to 0.
- The simple solution is to use weakened refs.
- If you don't do this, the memory will not be freed until the program is terminated.

Now, the code I'm dealing with certainly contains circular references. But before I start looking up all those and weakening them, I wanted to understand something:

If I reach the end of my program, and it terminates as it should - is the memory with the circular references freed anyway? If it is, does it mean that technically, there can be no such thing as a memory leak in perl as long as the program isn't interrupted? What happens if it is interrupted? I couldn't find definite answers to these questions...

Thanks a lot!

Replies are listed 'Best First'.
Re: Memory and garbage collection
by moritz (Cardinal) on Jul 30, 2008 at 12:50 UTC

    The four points you listed are correct.

    When a process dies or exists, all its memory is freed. The same holds true for perl programs, because on the operating system level they are just ordinary programs.

    The term "memory leak" refers to the fact that a long running program can use more and more memory, without actually storing more information. This happens when memory is not freed, but the program has no references to that piece of memory.

    What happens if it is interrupted?

    What do you mean by "interrupted"? The scheduler of a multi tasking operation system interrupts your program many times each second, and resumes it afterwards. But it abstracts that away, the process doesn't "feel" that it has been interrupted.

[reply]

      Oh, sorry, I meant killed, not interrupted. But I guess that brings me back to the first thing you said ("When a process dies [...]").

      I'm working on a 64-bit linux machine, and I tried watching what happens to the used memory before, after and during the run of my program. After the program exits, the amount of used memory goes down, but not to what it was before I ran the program (and my process is the only one on that machine that uses a lot of memory....). So now, what I don't understand is - if the OS frees all the program's memory - why does it happen?

[reply]
        Linux has the philosophy "if you need free memory, put it beside your PC". Or in other words, it buffers as much as it can, and doesn't show the buffered as "free" just for the sake of having free memory available.

        For example it caches all files read from disk, like your script, the perl interpreter, modules and other data files. So after you run a perl program linux will say it has less free memory. But if you need the memory, it clears the caches and makes that memory available.

        So after running a perl program, or any kind of program, you can have less free memory than before, but you have no loss whatsoever.

        Try running the same program 20 times in row. Memory usage shouldn't increase for each run, just for the first one.

[reply]
        If your memory is less, there are a few possibilities:

        a) your program uses a GUI. X11 might allocate memory for your program which it isn't giving back

        b) your program doesn't really exit. Check your system for defunct processes

        c) some OS caches might fill up memory. Did you for example access the file system with your script?

        d) your OS or some other process interacting with your script might have a memory leak

        You might save the output of 'ps -elf' to some file between a few starts of your script and see if you can spot any differences in the memory footprint of the processes. If not, either the differences are too small or your OS is using up the memory.

[reply]
Re: Memory and garbage collection
by dragonchild (Archbishop) on Jul 30, 2008 at 13:25 UTC
    A memory leak is where memory is allocated by the process, but is unreachable and unusuable. This means that when the memory would normally have been released back to the pool of allocatable memory, it isn't. Thus, a loop that would normally consume a given amount of memory and no more now consumes a little more each time (ie, it's leaking).

    Only processes leak memory. When a process ends, the OS reclaims that memory and the process, since it's finished, has nothing to say on the matter. Processes only borrow memory from the OS. When they're done, the OS gets it back.

    Does that help?

    My criteria for good software:
    1. Does it work?
    2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?
[reply]
Re: Memory and garbage collection
by pjotrik (Friar) on Jul 30, 2008 at 13:10 UTC
    If I reach the end of my program, and it terminates as it should - is the memory with the circular references freed anyway? If it is, does it mean that technically, there can be no such thing as a memory leak in perl as long as the program isn't interrupted?
    If by "interrupted" you mean killed (e.g. by ctrl-c), it makes no difference. Memory allocated by the process is returned to the system when the process terminates, regardless of how it terminates. Memory leaks are only happening during the run of a program, usually they're encountered with long-running programs like daemons.
[reply]
Re: Memory and garbage collection
by Limbic~Region (Chancellor) on Jul 30, 2008 at 13:33 UTC
    Anonymous Monk,
    The question about how perl uses memory and if it returns it to the OS comes up from time to time. The answer is always - "it depends". See this node in which I have a number of links which might help.

    As has already been answered by others, when perl terminates, the memory is returned back to the OS. You haven't mentioned what your application is, but if it is mod_perl for instance it might be a horse of a different color. Additionally, intentional circular references are an oddity for sure - what are you using them for? We can probably suggest a better way.

    Cheers - L~R

[reply]
Re: Memory and garbage collection
by Khen1950fx (Canon) on Jul 30, 2008 at 13:21 UTC
    Your question started me thinking...I couldn't answer the questions; however, I did find an article about customized proxy objects that made things a little clearer for me. See:

    Proxy Objects by Matt Sergeant.

[reply]
      I actually read the same article earlier today :) It does explain things very clearly.
[reply]

Back to Seekers of Perl Wisdom