in reply to Massive Perl Memory Leak

1) Don't use & when calling subroutines. And if you do, DO MAKE SURE you do not use it when calling a subroutine with no parameters! Unless you know what that means.

2) Don't use 

%{$datahash{"devinfo"}} = %devinfo;

[download]
use 
$datahash{"devinfo"} = \%devinfo;

[download]
instead. It's silly to force perl to flatten the %devinfo hash into a list, build a new has containing the data and then throw %devinfo out of window. Just take a reference.

3) Don't use the global. Pass a reference to the %datahash 

for (;;) {
 my %datahash;
 ..getdevicetopoll..
 ifpoll( \%datahash, $dev);
 print FILE dump %datahash;
}
...
sub ifpoll {
 my ($datahash, $dev) = @_;
 my %devinfo;
 my %interfaces; ## etc...
 ..snmp a lot of data here..
 $datahash->{"devinfo"} = \%devinfo;
 $datahash->{"interfaces"} = \%interfaces;
 return;
}

[download]

Neither of those changes should affect the leak though, it must be somewhere else in the code. Are you sure you are not producing any cyclic references in the part of the code you did nto show us?

Jenda
Support Denmark!
Defend the free world!

Replies are listed 'Best First'.
Re^2: Massive Perl Memory Leak
by wagnerc (Sexton) on Jun 13, 2007 at 19:37 UTC
    Hi. When dealing with my own subs I always use the &name(); form. That way everything is made explicit and differentiating the subs from the builtins is easier. Wouldn't re-my'ing datahash in ifpoll break the reference back the caller's datahash?

    Oh so Perl flattens hash assignments? I thought it was smart enough to know u were assigning a hash to a hash.

    UK: The problem with validating as single threaded is that the leak is obscured in Perl's normal bloat. And the number of iterations needed to show a noticable mem increase would take a *long* time. Even with 100 threads it doesn't go into the death spiral for 20-30 minutes. But I know what ur saying.

[reply]

      1) Well, if you feel better using the & go ahead, just keep in mind that &foo(); and &foo; are two very different things. The first statement calls the foo() subroutine with no parameters while the second one calls it aliasing its @_ to the current @_. That is any changes to @_ made within foo() will affect the current subroutine!

      2) No, the my affects only the variable, no its contents. So the ifpoll() receives a reference to the caller's %datahash and assigns the reference to its own variable $datahash. The my itself doesn't affect the %datahash in any way.

      3) Per has to flatten hash assignments. It might make some optimizations like preallocating the right number of buckets, but it has to create a new hash anc copy the values. So that changing one hash doesn't affect the other.
      The %newhash = %oldhash; makes a copy, not another name for the same "object".

      Jenda
      Support Denmark!
      Defend the free world!

[reply]
[d/l]
[select]
        but it has to create a new hash anc copy the values. So that changing one hash doesn't affect the other.
        Sort of. As we found out in other posts only the first dimension is truly copied. With a hash of hashes the sub hashes come over as references and both hashes refer to the same subhash references.
[reply]

In Section Seekers of Perl Wisdom