in reply to Grabbing Webpages with Usernames and Passwords

It is very simple to get a protected page using LWP::UserAgent. A little example:

use LWP::UserAgent; use HTTP::Request; use HTTP::Cookies; # setup your browser $ua = LWP::UserAgent->new(keep_alive => 1, timeout => 300); # what kind of browser you are $ua->agent("Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.3a) Gecko/200 +21207 Phoenix/0.5"); # a place to store cookies, if needed $ua->cookie_jar( HTTP::Cookies->new(file => "perlcookies.dat", autosav +e => 1) ); # now build a request $request = HTTP::Request->new( GET => $url ); # set credentials $request->authorization_basic($username, $password); # run browser $response = $ua->request($request); # and check response if ($response->is_success) { print $response->content, "\n"; } else { die "failed: ", $response->message, "\n"; }

Happy downloading :) Valerio

Replies are listed 'Best First'.
Re: Re: Grabbing Webpages with Usernames and Passwords
by cdherold (Monk) on Feb 20, 2003 at 22:13 UTC
    I tried that code ... what does this error mean?

    Can't locate MIME/Base64.pm in @INC (@INC contains: /usr/lib/perl5/5.6 +.0/i386-linux /usr/lib/perl5/5.6.0 /usr/lib/perl5/site_perl/5.6.0/i38 +6-linux /usr/lib/perl5/site_perl/5.6.0 /usr/lib/perl5/site_perl .) at + /usr/lib/perl5/site_perl/5.6.0/HTTP/Headers.pm line 588.

      You must install module MIME::Base64. Download it from the previous link, unpack it and follow instructions contained in README. If you want a simpler way, just fire up a CPAN shell (perl -MCPAN -e shell) and type install MIME::Base64.

      Ciao, Valerio

      It means you need to install the MIME::Base64 module, base64 is used to encode the username and password when accessing web pages protected with http basic auth.