Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

Verify web proxy servers at www.publicproxyservers.com

by merlyn (Sage)
on Mar 22, 2006 at 05:09 UTC ( [id://538412]=CUFP: print w/replies, xml ) Need Help??

In response to using proxy servers with lwp::simple or useragent, I decided to try a proxy tester combined with a XML::LibXML screen scraper. Works pretty well, if you have a few minutes to run the test. Please change "www.example.com" to an appropriate probe URL.
#!/usr/bin/perl use strict; use XML::LibXML; use LWP::UserAgent; my $URL = "http://www.example.com/"; my $ua = LWP::UserAgent->new; $ua->timeout(5); my $parser = XML::LibXML->new; $parser->recover(1); for my $count (1..9) { my $doc = do { local *STDERR; open STDERR, ">/dev/null"; $parser->parse_html_file("http://www.publicproxyservers.com/page$c +ount.html"); }; for my $row ($doc->findnodes("//b[. = \"IP\"]/../../../tr")) { my ($host, $port) = ($row->findvalue("td[1]"), $row->findvalue("td +[2]")); next unless $host =~ /^\d/; print "$host:$port... "; $ua->proxy("http", "http://$host:$port/"); my $res = $ua->get("http://www.example.com"); print $res->status_line, "\n"; } }

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: CUFP [id://538412]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others having a coffee break in the Monastery: (2)
As of 2024-04-26 01:29 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found