in reply to Serious memory leak when using LWP

I think you're blaming the wrong module (LWP).  When I take HTML::Parse out of the game, I don't get any leakage.

OTOH, when using just HTML::Parse, I see it leaking, too.

use HTML::Parse;

while(1) {
    parse_html('');

    system "ps -p $$ -o rss,vsz";
    sleep 1;
}

[download]

(Not that this makes much difference from an end user perspective... but you now at least know what to file a bug report against.)

The usual workaround for long running programs with memory leaks is to fork a new worker process every n iterations.

Replies are listed 'Best First'.
Re^2: Serious memory leak when using LWP
by Anonymous Monk on Mar 08, 2012 at 04:33 UTC

    FWIW

    HTML::Parse returns a HTML::Tree object

    Its a known limitation of HTML::Tree, you have to delete() the tree to free the memory, otherwise it will leak a LOT of memory (circular references are circular, keep objects alive forever, unless they're weak)

    Rejected in 2005/06 Bug #12283 for HTML-Tree: Use weaken to avoid ->delete()?

    With delete there doesn't seem to be much of a leak 

    perl -MHTML::Parse -le " system qq[pslist.exe -m $$ 2>NUL]; for(1..400
+00){ $ju = parse_html(q{ <p>yo</p> }); $junk = $ju->format; $ju->dele
+te; } system qq[pslist.exe -m $$ 2>NUL]; "

    [download]
[reply]
[d/l]
      If you notice that any package provides an explicit destructor method ... call it what you will ... USE it. "There must be a reason." If it doesn't, still explicitly set references to "undef" instead of just letting them go out of scope. Is this advice necessary? I don't know.
[reply]

        delete() is not a destructor, the destructor never gets called because the reference count is not zero

[reply]

In Section Seekers of Perl Wisdom