TheFlyingCorpse has asked for the wisdom of the Perl Monks concerning the following question:

Hello, I am trying to write a small script that logs in to a webpage that is protected with a Basic / Digest authentication. I am however unable to log in via WWW::Mechanize and LWP::UserAgent. I've tried these code examples, without avail.

1: 

my $username = "someuser";
my $password = "password";
my $url = 'http://sub.example.com:49282/navigation/nav_home.shtml';

use LWP::UserAgent;
$ua = LWP::UserAgent->new;
$req = HTTP::Request->new(GET => $url);
$req->authorization_basic($username,$password);
print $ua->request($req)->as_string;

[download]

2: 

## LWP!
use LWP;
use strict;

my $username = 'username';
my $password = 'password';
my $url = 'http://server:49282/subpage/nav_home.shtml';

my $browser = LWP::UserAgent->new('Mozilla');
$browser->credentials("server:49282","Digest realm",$username=>$passwo
+rd);
my $response=$browser->get($url);

print $response->content();

[download]

3: 

## WWW::Mechanize
#!/usr/bin/perl

use WWW::Mechanize;

my $username = 'username';
my $password = 'password';
my $url = 'http://server:49282/subpage/nav_home.shtml';
# Create a new instance of WWW::Mechanize
my $mechanize = WWW::Mechanize->new(autocheck => 1);

# Supply the necessary credentials
$mechanize->credentials("server:49282","Digest realm", $username=>$pas
+sword);
$mechanize->get($url);
# Retrieve the desired page
print $mechanize->content();

[download]

The site I try to log in to should not have any fancy mechanics to avoid being logged into via Scripts. ANy ideas or examples of how I can log in? From what I've read up on Basic authentication, it can't authenticate until it has got the WWW-Authorization header from the server, which is what I've loaded up the correct credentials for. I've also tried to load the page, do the "reply" in terms of credentials, without success.

Replies are listed 'Best First'.
Re: Basic / Digest authentication in website
by alexbio (Monk) on Jul 11, 2010 at 16:45 UTC

    The information you posted is not sufficient, can you please tell also what you get using the code you showed?

    I usually use LWP::UserAgent for GET request with authentication as follows:

    use LWP::UserAgent;
use HTTP::Request::Common;

my ($url, $usr, $pwd, $endpoint, $realm) = @_;
my $ua = LWP::UserAgent -> new;
    
$ua -> credentials($endpoint, $realm, $usr, $pwd);
my $response = $ua -> request(GET $url) -> as_string;

    [download]

    That is similar to the second code block you posted.

    Update: It could be useful if you also post the real URL of the file you are trying to get as http://sub.example.com:49282/navigation/nav_home.shtml and http://server:49282/subpage/nav_home.shtml are not real addresses.

    Alex's Log - http://alexlog.co.cc
[reply]
[d/l]
[select]
      The first block of code gives me this output, which also the code example you posted gives me:

      HTTP/1.1 401 Unauthorized
      WWW-Authenticate: Digest realm="GWAVA Console",qop="auth",nonce="d951626547bbe35 ffba6032ba46959bc",opaque="38f98a915296a94de4bbdaa09cb94726"
      Client-Date: Sun, 11 Jul 2010 15:51:19 GMT
      Client-Peer: 10.10.10.10:49282
      Client-Response-Num: 1

      The URL is this: http://mail.isnitro.com:49282/navigation/nav_home.shtml (It is only reachable on the internal network)
[reply]
        Then find a publicly available site where your problem occurs.
[reply]
Re: Basic / Digest authentication in website
by zwon (Abbot) on Jul 11, 2010 at 16:31 UTC

    I'd use wireshark to see what happens. Unfortunately you have not provided us with most interesting data -- the responses you've got.

[reply]
      THe first block of code gives me this output, which is very similar to all theothers regarding the 401.

      HTTP/1.1 401 Unauthorized
      WWW-Authenticate: Digest realm="GWAVA Console",qop="auth",nonce="d951626547bbe35 ffba6032ba46959bc",opaque="38f98a915296a94de4bbdaa09cb94726"
      Client-Date: Sun, 11 Jul 2010 15:51:19 GMT
      Client-Peer: 10.10.10.10:49282
      Client-Response-Num: 1

[reply]

        Your third piece of code:

        $mechanize->credentials("server:49282","Digest realm", $username=>$pas
+sword);

        [download]

        The server's response:

        WWW-Authenticate: Digest realm="GWAVA Console", ...

        [download]

        Do you see the important difference?

        Either use the two-argument form of WWW::Mechanize->credentials() or fix the realm argument of the four-argument form to match the server's 401 response.

        The second piece of code has the same problem, but you can't use the two argument form of LWP::UserAgent->credentials() (it is a getter, not a setter).

        zwon explained you why the first piece of code doesn't work.

        Alexander

        --
        Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
[reply]
[d/l]
[select]

        Ok, so it is Digest authentication, not Basic, that's why the first block of code doesn't work for you.

[reply]
        The realm is "GWAVA Console", but you told LWP to only respond to the realm "Digest realm"
[reply]
Re: Basic / Digest authentication in website
by rowdog (Curate) on Jul 11, 2010 at 21:37 UTC

    LWP handles digest authentication as long as you get the credentials right. A simple example... 

    #!/usr/bin/perl -T
use strict;
use warnings;

use LWP::UserAgent;

my $ua = LWP::UserAgent->new;
$ua->credentials('jigsaw.w3.org:80', 'test', 'guest', 'guest');
my $response = $ua->get('http://jigsaw.w3.org/HTTP/Digest/');
$response->is_success or die $response->status_line;
print $response->decoded_content;

    [download]

    You can check your credentials with Firefox. Open the URL, notice the realm it tells you, enter user/pass and verify that you can actually log in. If all that checks out, you might need to deal with cookies and/or Javascript.

[reply]
[d/l]

Back to Seekers of Perl Wisdom