Hello, I'm currently coding a web crawler and I've been using WWW::Mechanize::Firefox to extract data from pages that keep loading content via JavaScript. Here's the code for this part of the script:

use strict;
use warnings;

use WWW::Mechanize::Firefox;     # Requires MozRepl addon for Firefox
use WWW::Mechanize;
use Date::Manip;
use DateTime;

open TXT, "<", "nickname.log";
while(! eof (TXT)){    
    my $nick = <TXT>;
    chomp $nick;
    
    my $j = 0;
    for $j(0..3){
        open MATCH, "+<", "match.log";
        my (@gotmatch) = <MATCH>;
        
        my ($date) = DateTime->now;
        
        my ($k) = 0;
        while($k < $j){
            $date = $date->subtract( days => 7);
            $k++;
        }
        
        $date = $date->ymd;
                        
        $url = "http://www.quakelive.com/#profile/matches/$nick/$date"
+;
        
        my ($firemech) = WWW::Mechanize::Firefox->new();
        $firemech->get($url);
        die "Cannot connect to $url\n" if !$firemech->success();
                
        my ($retries) = 10;
            while ($retries-- and ! $firemech->is_visible( xpath => '/
+/*[@class="areaMapC"]' )) {
                  sleep 1;
            }
            die "Timeout" unless $retries;

        my ($content) = $firemech->content();

        while(($content =~ /class="areaMapC" id="([^<]+)_([^<]+)_([^<]
++)">/gsi)){
            my ($game) = $1;
            my ($longid) = $2;
            my ($shortid) = $3;
            my ($matchid) = "$longid/$game/$shortid\n";
            
            # Checks match.log for duplicates
            my ($flag) = 0;
            for my $l(0..$#gotmatch){
                if($matchid eq $gotmatch[$l]){
                    $flag = 1;
                }
            }
            if($flag == 0){
                print MATCH $matchid;
                $flag = 0;
            }
            
        }
        close MATCH;
        undef $firemech;
        $j++;    
    }
}

[download]

The rest of the script (using WWW::Mechanize) runs perfectly, but this part breaks it giving these errors:

(in cleanup) Can't call method "cmd" on an undefined value at /Library
+/Perl/5.10.0/MozRepl/Client.pm line 186,<DATA>  line 42 during global
+ destruction.
    
    (in cleanup) Can't call method "execute" on an undefined value at 
+/Library/Perl/5.10.0/MozRepl.pm line 372,<DATA>  line 42 during globa
+l destruction.

    (in cleanup) Can't call method "is_debug" on an undefined value at
+ /Library/Perl/5.10.0/MozRepl/Client.pm line 188,<DATA>  line 42 duri
+ng global destruction.

[download]

Note that it sometimes gives only one of them, sometimes all of them and that "is_debug" is the most rare of them. Also note that the username they pop out is different every run (sometimes the 6th, others the 15th, etc).

On StackOverflow I've been told to "undef $firemech", but this didn't solve the issue. Google hasn't been helpful either so far, nor has been my Perl professor.

Thanks.

In reply to MozRepl cleanup problem by giantpanda

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.