Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I'm having problems getting web pages using LWP:Simple. When I'm logged into the site and paste the url into the browser, the page comes up just fine. But when I pass the same url to 'get' command, I get the site's login page. I've used the 'get' command many times in the past and I have the username/password to this site -- is there any way to get the pages that I need instead of the login page? Has anyone worked around this?

Replies are listed 'Best First'.
Re: LWP:Simple bypass problem
by davidrw (Prior) on Jun 05, 2006 at 20:33 UTC
    You could just use WWW::Mechanize (it uses LWP::Simple underneath) to login properly .. it could also be that the server is bouncing you based on your browser, in which case you can fudge the user agent string. 
             use LWP::UserAgent;
         $ua = LWP::UserAgent->new;
         $ua->agent("MyApp/0.1 ");

# OR:

  use WWW::Mechanize;
  my $mech = WWW::Mechanize->new( agent=>"wonderbot 1.01" );

    [download]
[reply]
[d/l]
Re: LWP:Simple bypass problem
by philcrow (Priest) on Jun 05, 2006 at 20:50 UTC
    I think you can get past basic auth by adding your username and password to the url: 
    $request = HTTP::Request->new(
  GET => ’http://user:passwd@www.example.com/'
);

    [download]
    But, my memory may have failed since I last did it.

    Phil

[reply]
[d/l]
Re: LWP:Simple bypass problem
by ioannis (Abbot) on Jun 05, 2006 at 21:13 UTC
    You need the credentials() method of LWP::UserAgent; if you do this often, sub-class it into your our own Agent.
[reply]

Back to Seekers of Perl Wisdom