in reply to Abnormal memory in some modules

Some Perl modules tie up memory and don't release it. There are various reasons for which this can happen, some valid and some invalid. You shouldn't blame Perl for this until you know why those modules tied up memory.

For Devel::Leak, when you call Devel::Lease::NoteSV it makes a table of all variables in use by Perl. Apparently when you call it again it doesn't get rid of the old table when it does so. You could report that as a bug. While a leak in Devel::Leak might be an amusing bug, it is hardly Perl's fault.

Other modules might do something like Memoize data from one call hoping to reuse it in the next. Even though memory usage goes up, that is not a bug. It is intentional behaviour. Sometimes it is inconvenient and becomes a problem - in which case complain to the author. Perl should not prematurely get rid of data that the programmer has told Perl to track.

In fact Perl has some intentional behaviour of this form. Lexical variables declared with my still tie up space after you return from a function even though that variable should never be accessed again. The point of this is to avoid having to take the performance hit of allocating space for the variable on your next pass - Perl just reuses the already allocated space.

There is known misbehaviour due to reference counting. But that is how Perl is documented to work, and it is the responsibility of people creating circular references to break them. Plus I don't often see people accidentally creating circular references.

The fact that you have not accidentally seen misbehaviour in your own modules is an indication that Perl is pretty good about not leaking memory. If you do find a memory leak and can identify what causes it to leak memory, and it is something that shouldn't, then that is a bug. But unless you have something specific, the report is not very useful for making Perl better, and likely isn't a problem with Perl.

Replies are listed 'Best First'.
Re^2: Abnormal memory in some modules
by piotr (Initiate) on Aug 17, 2004 at 13:59 UTC
    Thanks for your enlightenment. I've been playing with perl -Dm compiled with -DDEBUGGING but I don't know yet how to make sense of the address it prints, or how do they map to perl data structures. 
    0x81cdb38: (01225) malloc 28 bytes
0x81cf880: (01226) malloc 28 bytes
0x81cdb58: (01227) malloc 40 bytes
0x81ce210: (01228) malloc 16 bytes
0x81c2560: (01229) free

    [download]
    Also I've found that the offending modules, although the objects I create inside the loop, are scoped inside the loop, so the object should be destroyed each loop iteration, as it happens with my test modules: 
    new Test
DESTROY Test
new Test
DESTROY Test
new Test

    [download]
    I have found the problem in Mail::Audit. It was a circular reference: 
    #   $self->{obj} = $self; # backwards-compatibility for everyone who d
+esperately needed access to $self

    [download]
    Taking that line out solves the problem. I'll send a notice to the author. Your insight has been very useful. Thanks.
[reply]
[d/l]
[select]
      In Mail::Audit you should not remove the line, instead if you're on Perl 5.6 or better, it should use WeakRef's weaken (shipped in the core of Perl from 5.6 on) to fix the reference counting.

      As for Devel::Leak, it may well use an internal data structure for this. If you're curious, peaking at sourcecode is helpful. In any case the problem is that Devel::Leak::NoteSV should clean up the results of having being called before if it is called again. If you can't puzzle out from the code how you might do this, the bug description you started with is specific enough for the author to probably figure it out.

      But just in case, it never hurts to attach a complete description of your environment to a bug report just in case it matters. perl -V will give you such a description.

[reply]
[d/l]

In Section Seekers of Perl Wisdom