using proxy servers with lwp::simple or useragent

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: using proxy servers with lwp::simple or useragent by sulfericacid (Deacon) on Mar 22, 2006 at 04:38 UTC
If you'd have done a search, I think you'd have found exactly what you were looking for. Take a look at LWP::Simple setting a proxy. If your proxies don't require authentication, this should work `$ENV{HTTP_proxy}="http://proxy.company.com:80";` [download] All you'd have to do is prepare your list in the array and loop over it while doing all the fun stuff inside. Hope this helps. "Age is nothing more than an inaccurate number bestowed upon us at birth as just another means for others to judge and classify us" sulfericacid	[reply] [d/l]
Re^2: using proxy servers with lwp::simple or useragent by Anonymous Monk on Mar 22, 2006 at 04:50 UTC
that's exactly what i was looking for!!! thanks man	[reply]
Re^3: using proxy servers with lwp::simple or useragent by Anonymous Monk on Mar 22, 2006 at 05:05 UTC
Actually with more testing it didn't work. Maybe it's working but I don't understand? Should this be hitting my web site under a new IP address every time? `use strict; use warnings; use LWP::Simple; my @list = qw(211.202.1.52:80 61.139.87.132:80 72.13.246.65:3128 203.1 +15.10.11:80 203.115.10.38:80); foreach my $prox (@list) { $ENV{HTTP_proxy}= $prox; my $url = "www.sensationalscentsonline.com"; print "$prox attempted..\n"; }` [download]	[reply] [d/l]
Re^4: using proxy servers with lwp::simple or useragent by merlyn (Sage) on Mar 22, 2006 at 05:14 UTC
Re: using proxy servers with lwp::simple or useragent by merlyn (Sage) on Mar 22, 2006 at 05:10 UTC
I got it to change with a simple `$ua->proxy('http', "http://$ip:$port");` [download] as demonstrated in Verify web proxy servers at www.publicproxyservers.com. -- Randal L. Schwartz, Perl hacker Be sure to read my standard disclaimer if this is a reply.	[reply] [d/l]
Re: using proxy servers with lwp::simple or useragent by polettix (Vicar) on Mar 22, 2006 at 10:49 UTC
Just curious, what's unclear in the docs of LWP::UserAgent? `$ua->proxy($scheme, $proxy_url) Set/retrieve proxy URL for a scheme: $ua->proxy(['http', 'ftp'], 'http://proxy.sn.no:8001/'); $ua->proxy('gopher', 'http://proxy.sn.no:8001/'); The first form specifies that the URL is to be used for proxying of access methods listed in the list in the first method argument, i.e. 'http' and 'ftp'. The second form shows a shorthand form for specifying proxy URL for a single access scheme.` [download] In your case, it seems that you can use the second, simplified form just like merlyn pointed out. Flavio perl -ple'$_=reverse' <<<ti.xittelop@oivalf Don't fool yourself.	[reply] [d/l]
Re: using proxy servers with lwp::simple or useragent by tweetiepooh (Hermit) on Mar 22, 2006 at 11:23 UTC
Just to add to what's gone before, if you need to authenticate on the proxy then the following may work. (This is not Perl code) `http_proxy=http://user:password@proxy:port` [download] I set environment http_proxy and ftp_proxy to the same value but note that active FTP may be blocked as may FTP as a whole. I have an account just for proxy access as user:password is world readable so unsafe to have data accessible anywhere else.	[reply] [d/l]