Re: Hitting a web page

Use LWP::UserAgent.

#!/usr/bin/perl -w

use strict;
use LWP::UserAgent;
use URI::URL;

my $ua = new LWP::UserAgent; 

my $uri = new URI::URL "http://www.perlmonks.org";
$uri->query_form("node_id" => "43958"); 

my $req = new HTTP::Request GET => $uri->as_string;

my $res = $ua->request($req);
 
if (!$res->is_success) {
  die 'Fetch failed';
}

my $content = $res->content;
print $content;
[download]

Comment on Re: Hitting a web page Download Code

Replies are listed 'Best First'.
Re: Re: Hitting a web page by tame1 (Pilgrim) on Nov 29, 2000 at 22:38 UTC
This seems to have worked, with some exceptions. (BTW - thanks!) I added `$ua->proxy(['http','ftp'],'http://my.proxyserver.com:83');` The only problem seems to be that my proxy requires a password, and reading throuhg perldoc LWP::UserAgent, I cannot find how I am supposed to pass the username and pass that the proxy server requires! I see how to pass them if the remote server needs them, but not the proxy ones. Are you at all familiar with this? What does this little button do . .`<Click>`; "USER HAS SIGNED OFF FOR THE DAY"	[reply] [d/l] [select]
Re: Re: Re: Hitting a web page by swiftone (Curate) on Nov 29, 2000 at 22:41 UTC
See RE: Getting a webpage through proxy, which has the following code: `use LWP::UserAgent; #..... usual stuff .....# $ua = new LWP::UserAgent; $ua->proxy(['http', 'ftp'] => 'http://your_proxy.address.com:'); my $req = new HTTP::Request 'GET', "http://www.perlmonks.org"; $req->proxy_authorization_basic("$user_name","$user_password"); $res = $ua->request($req);` [download] (Obtained through Super Search for "LWP proxy password". Super Search is also your friend!)	[reply] [d/l]
Re: Re: Hitting a web page by tame1 (Pilgrim) on Nov 29, 2000 at 22:45 UTC
Disregard my last - I did some searching on the links that one of the other responders had posted (thanks folks) and discovered `$req->proxy_authorization_basic` Again, thanks everyone. What does this little button do . .`<Click>`; "USER HAS SIGNED OFF FOR THE DAY"	[reply] [d/l] [select]