As far as I can see from a simple test, perl -e 'open(BLA,">>/tmp/bla"); while(<BLA>){print}'

The test shows that won't give you an error messages, but it will exit the loop immediately.

Which is why your webcrawler exits prematurely, without even reaching the calls to links. You should read from a different file than you are writing to.

(And you should step through it in debugger to see if what you think is happening is actualy happening.)

use strict;
use warnings;
use Data::Dumper;
use WWW::Mechanize;

my $a = WWW::Mechanize->new();
$a->get( 'http://some.site.com.' );

print $_->url,"\n" for @{ $a->links };
[download]

L-LL-L--L-LL-L--L-LL-L--
-R--R-RR-R--R-RR-R--R-RR
B--B--B--B--B--B--B--B--
H---H---H---H---H---H---
(the triplet paradiddle with high-hat)

does www::mechanize follow the rules set by lwp::robotua? I know that before ifi tried to "get" something that a robots.txt file didnt allow me too, my get came up empty (a good thing). Does www:mechanize allow me to do the same thing? I have been to CPAN and looked at it, but didnt see anything about obeying rules. Thanks

Kudos to you for wanting polite bots. The problem with getting LWP::RobotUA to play nice with WWW::Mechanize is that they both are subclasses of LWP::UserAgent. By itself, WWW::Mechanize does not consult the /robots.txt file, but you can instead use WWW::RobotRules. Here is a working example that tries to grab two files from my server:
Read more... (856 Bytes)
There might be a better way though ... ahh, how about "WWW::Mechanize::Polite"?
Read more... (1364 Bytes)
And if i didn't just reinvent a wheel, you might be seeing this on the CPAN. ;)

jeffa

L-LL-L--L-LL-L--L-LL-L--
-R--R-RR-R--R-RR-R--R-RR
B--B--B--B--B--B--B--B--
H---H---H---H---H---H---
(the triplet paradiddle with high-hat)

$url = new URI::URL $links, $content->base();
print LINKS $url->as_string(), "\n";
[download]