Dear Perl Monks,

There is a memory leak in the following code. I am reading in 28 000 website adresses from one file, then run them through a while loop, extracting some information from them. From what I have read at other nodes, the problem probably lies with an internal call of HTML::FormatText to HTML::Treebuilder, which creates circular references which do not let the variables containing the formatted text of the website scope out when leaving the while-loop.

 
use warnings;
use strict;
use diagnostics;
use HTML::FormatText;
use HTML::TreeBuilder 5 - weak;
open INPUT, "< D:/websiteadresses.txt" or die "Problem: $!";
my @INPUT=<INPUT>;
close INPUT;
while (@INPUT) {
 my $input=shift(@INPUT);   
  chomp $input;
 my $content=HTML::FormatText->format_file($input, leftmargin => 0, ri
+ghtmargin => 50);
 # followed by some regular expressions, the results of which are save
+d in a new file - all of this is commented out at the moment
}
[download]

According to the earlier posts this

 
use HTML::TreeBuilder 5 - weak;
[download]

should take care of it - but it does not. The program still runs out of memory after 3000 runs through the while loop

I am now trying to use Object::Destroyer to clear up all objects created by the call to HTML::FormatText. However, I do not understand the relationship between HTML::FormatText and HTML::Treebuilder. The following code will not execute, giving the error message "You should pass an object or constructor". What am I doing wrong here?

 
use warnings;
use strict;
use diagnostics;
use HTML::FormatText;
use HTML::TreeBuilder 5 - weak;
open INPUT, "< D:/websiteadresses.txt" or die "Problem: $!";
my @INPUT=<INPUT>;
close INPUT;
while (@INPUT) {
 my $input=shift(@INPUT);   
  chomp $input;
 my $content=HTML::FormatText->format_file($input, leftmargin => 0, ri
+ghtmargin => 50);
 my $sentry = Object::Destroyer->new($content, 'delete' );
 # followed by some regular expressions, the results of which are save
+d in a new file - all of this is commented out at the moment
}
[download]

I have also tried this code, based on an earlier node. It executes properly, but still has a memory leak.

use warnings;
use strict;
use diagnostics;
use HTML::FormatText;
use HTML::TreeBuilder; 
use Object::Destroyer;
open INPUT, "< D:/websiteadresses.txt" or die "Problem: $!";
my @INPUT=<INPUT>;
close INPUT;
while (@INPUT) {
 my $inputfile=shift(@INPUT);   
 chomp $inputfile;
 print $inputfile;
 my $formatter = HTML::FormatText->new();
 my $content=$formatter->format(Object::Destroyer->new(HTML::TreeBuild
+er->new_from_file($inputfile), 'delete'));
}
[download]

Similarly, this (based on http://www.perl.com/pub/2007/06/07/better-code-through-destruction.html) also still has a memory leak;

use strict;
use HTML::TreeBuilder;
use Object::Destroyer 2.0;
open INPUT, "< D:/websiteadresses.txt" or die "Problem: $!";
my @INPUT=<INPUT>;
foreach my $filename (@INPUT) {
        chomp $filename;
        print $filename;
        my $tree   = HTML::TreeBuilder->new;
        my $sentry = Object::Destroyer->new($tree, 'delete');
        $tree->parse_file($filename);
    }
[download]

How can I properly destroy the objects created by the HTML::FormatText call to prevent the memory leak?

Any help would be greatly appreciated!

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.