in reply to Grabbing a hundred pages
So, only a little more complicated than LWP::Simple:
And the parse the HTML as described above.use LWP::UserAgent; use HTTP::Request::Common qw(GET); my $ua = LWP::UserAgent->new; my $response = $ua->request(GET $rdf->{rdf_url}); my $html = $response->content;
If you do wind up using LWP::Simple, check out the
construct so you are only processing html docs you haven't previously processed.is_success(mirror($URL, $URL_MIRROR)) { ...
|
|---|