perlseeker5 has asked for the wisdom of the Perl Monks concerning the following question:

Hi, I'm using a 3rd party package in my long running perl program that is leaking memory, I properly (I think) remove the variable that contains the object when I'm done with it but it's memory does not free. Consider this example:

my $hash = {1 => 2, 3 => 4};
my $retval = \$hash;

I've been using Devel::Peek so I can see $hash has a refcount of 2. Now all I have access to is $retval in my program, how can I decrease the reference count of $hash? I haven't seemed to be able to figure out how to use SvREFCNT_dec correctly in this situation. Of course $retval has a refcount of 1, so it is correctly free'd but $hash is not because it has a refcount of 2.

This is a simplified example of course, the 3rd party module I am using gives me a hash reference that probably has circular references in it which is why it doesn't get garbage collected properly.

Thanks for any help! I can of course just wrap this program in another that restarts it whenever it's memory usage gets to be to much, but I thought I would ask if this method was available.

Replies are listed 'Best First'.
Re: reduce reference count of hash
by moritz (Cardinal) on Feb 07, 2008 at 23:11 UTC
    The refcount of $hash is 2 because $retval and $hash both reference it.

    The method you are looking for could be Scalar::Util::weaken.

    But that's unlikely to help if your third party module has cyclic references in internal data structures. Try Devel::Leak or Devel::Leak::Object to find the bad guy.

Re: reduce reference count of hash
by Joost (Canon) on Feb 08, 2008 at 00:28 UTC
    Up front: moritz's post seems to be correct (but I'm tired).

    how can I decrease the reference count of $hash?
    Why?

    Of course $retval has a refcount of 1, so it is correctly free'd but $hash is not because it has a refcount of 2.
    Yes. At least if you're decrementing the refcount on $retval. Decrementing the refcount of $hash will result in all kinds of mahem.
    This is a simplified example of course, the 3rd party module I am using gives me a hash reference that probably has circular references in it which is why it doesn't get garbage collected properly.

    Circular references wreak havoc on perl's GC, and have to be broken up before releasing. That's a well known issue, and I'd bet a case of good beer on it not being fixed ever in perl versions < 6 (and perl6 will change a lot of scope semantics anyway).

    None of this has anything to do with the code you posted. In other words: what are you talking about?

Re: reduce reference count of hash
by CountZero (Bishop) on Feb 08, 2008 at 08:50 UTC
    It is difficult to give a good advice as we do not know which "3rd party module" your are referring to.

    On the other hand, it shouldn't be too difficult to trace all references to that hash reference inside that "3rd party module", at least if it is written in Perl.

    As a last ditch desperate measure, you could weaken the hash reference that the module returns to you (just before it is returned of course, for which you will need access to that 3rd party module), although it is likely that this will introduce all kinds of subtle bugs.

    CountZero

    A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

Re: reduce reference count of hash
by chromatic (Archbishop) on Feb 08, 2008 at 02:37 UTC
    I properly (I think) remove the variable that contains the object when I'm done with it but it's memory does not free.

    How are you measuring that? If you're looking at the Task Manager or running top, your application probably won't give memory back to the operating system.

      Yes using top, but it keeps growing and then gets killed by OS after running for a while. Then using backticks I narrowed it down to the code segment that seemed to be causing it, and since it seems like a suspicious part of the code in that it's the only part that uses this 3rd party module I thought it might be it and investigated further which lead to my question about how to reduce the ref count of a hash that I only have a reference to.

      Thanks very much for all your replyes!
Re: reduce reference count of hash
by locked_user sundialsvc4 (Abbot) on Feb 08, 2008 at 22:43 UTC

    When you are considering curiosities like this, be very careful to properly consider what the “reference count” you are looking at belongs to. For instance, in your example there are three players in the game:

    1. The variable "$hash."
    2. The variable "$retval."
    3. A hash... being a value, floating in space.

    To Perl's memory-manager, all three of these separate things are distinct. The lifetime of variables is, of course, determined by their scope, but “a variable” is distinct from “the value it ‘contains.’” To put it another way, a variable holds a reference to the thing that it contains, until it (the variable) goes out-of-scope.

    If you set variable X to be “a reference to” another variable Y, then change the value of the referenced variable Y to some other value, you will observe that X continues to be a reference to the value that Y used to contain. That value continues to exist, because Y holds a reference to it.

    References are a powerful part of Perl, but they can be confusing in-part because there is a certain amount of ambiguity:   the syntax implies that the reference is “to a variable,” when it's really a reference to a value. You don't usually think of values as being “things,” but Perl does.