Problems with WWW::Mechanize

robmarano has asked for the wisdom of the Perl Monks concerning the following question:

Hey Monks,

I have been sweating over this issue for 2 days: For some reason when I use credentials in the following way, the debug states the agent is unauthorized for use even after I set up the credentials as per instructions:

#!/usr/bin/perl
use warnings;
use strict;
use LWP::Debug qw( + );
use WWW::Mechanize;
use CGI;
use MIME::Base64;
use HTTP::Cookies;

my $netloc = "10.4.4.4:8080";
my $url = "http://" . $netloc . "/admin";
my $realm = "Realm";
my $username = "user";
my $password = "pass";

my $cookie_jar = HTTP::Cookies->new(
    file => 'cookies.dat',
    autosave => 1,
);
my $agent = WWW::Mechanize->new(
    autocheck => 1,
    quiet => 0,
    agent_alias => 'Mac Mozilla',
    cookie_jar => $cookie_jar,
);

# Supply the necessary credentials
$agent->credentials($netloc, $realm, $username, $password);
$agent->get($url);

# Throw error if page not found
$agent->success() or die "Can't fetch the Requested page";
[download]

The output is as follows:

LWP::UserAgent::new: ()
LWP::UserAgent::request: ()
LWP::UserAgent::send_request: GET http://10.4.4.4:8080/admin
LWP::Protocol::http::request: ()
LWP::UserAgent::request: Simple response: Unauthorized
LWP::UserAgent::request: ()
HTTP::Cookies::add_cookie_header: Checking 10.4.4.4 for cookies
HTTP::Cookies::add_cookie_header: Checking .4.4.4 for cookies
HTTP::Cookies::add_cookie_header: Checking 4.4.4 for cookies
HTTP::Cookies::add_cookie_header: Checking .4.4 for cookies
HTTP::Cookies::add_cookie_header: Checking 4.4 for cookies
HTTP::Cookies::add_cookie_header: Checking .4 for cookies
LWP::UserAgent::send_request: GET http://10.4.4.4:8080/admin
LWP::Protocol::http::request: ()
LWP::UserAgent::request: Simple response: Unauthorized
LWP::UserAgent::request: ()
HTTP::Cookies::add_cookie_header: Checking 10.4.4.4 for cookies
HTTP::Cookies::add_cookie_header: Checking .4.4.4 for cookies
HTTP::Cookies::add_cookie_header: Checking 4.4.4 for cookies
HTTP::Cookies::add_cookie_header: Checking .4.4 for cookies
HTTP::Cookies::add_cookie_header: Checking 4.4 for cookies
HTTP::Cookies::add_cookie_header: Checking .4 for cookies
LWP::UserAgent::send_request: GET http://10.4.4.4:8080/admin
LWP::Protocol::http::request: ()
LWP::UserAgent::request: Simple response: Unauthorized
LWP::UserAgent::request: ()
HTTP::Cookies::add_cookie_header: Checking 10.4.4.4 for cookies
HTTP::Cookies::add_cookie_header: Checking .4.4.4 for cookies
HTTP::Cookies::add_cookie_header: Checking 4.4.4 for cookies
HTTP::Cookies::add_cookie_header: Checking .4.4 for cookies
HTTP::Cookies::add_cookie_header: Checking 4.4 for cookies
HTTP::Cookies::add_cookie_header: Checking .4 for cookies
LWP::UserAgent::send_request: GET http://10.4.4.4:8080/admin
LWP::Protocol::http::request: ()
LWP::UserAgent::request: Simple response: Unauthorized
LWP::UserAgent::request: ()
HTTP::Cookies::add_cookie_header: Checking 10.4.4.4 for cookies
HTTP::Cookies::add_cookie_header: Checking .4.4.4 for cookies
HTTP::Cookies::add_cookie_header: Checking 4.4.4 for cookies
HTTP::Cookies::add_cookie_header: Checking .4.4 for cookies
HTTP::Cookies::add_cookie_header: Checking 4.4 for cookies
HTTP::Cookies::add_cookie_header: Checking .4 for cookies
LWP::UserAgent::send_request: GET http://10.4.4.4:8080/admin
LWP::Protocol::http::request: ()
Error GETing http://10.4.4.4:8080/admin: Unauthorized at /Library/Perl
+/5.8.8/LWP/Authen/Basic.pm line 56
[download]

Then I add in the basic authentication information via MIME::Base64::encode AND tack that info to the $agent->get() call as follows AND IT WORKED!

#!/usr/bin/perl
use warnings;
use strict;
use LWP::Debug qw( + );
use WWW::Mechanize;
use CGI;
use MIME::Base64;
use HTTP::Cookies;

my $netloc = "10.4.4.4:8080";
my $url = "http://" . $netloc . "/admin";
my $realm = "Realm";
my $username = "user";
my $password = "pass";
my @args = (
        Authorization => "Basic " .
            MIME::Base64::encode( $username . ':' . $password )
);

my $cookie_jar = HTTP::Cookies->new(
    file => 'cookies.dat',
    autosave => 1,
);
my $agent = WWW::Mechanize->new(
    autocheck => 1,
    quiet => 0,
    agent_alias => 'Mac Mozilla',
    cookie_jar => $cookie_jar,
);

# Supply the necessary credentials
$agent->credentials($netloc, $realm, $username, $password);
$agent->get($url,@args);


# Throw error if page not found
$agent->success() or die "Can't fetch the Requested page";
[download]

Output:

LWP::UserAgent::new: ()
LWP::UserAgent::request: ()
HTTP::Cookies::add_cookie_header: Checking 10.4.4.4 for cookies
HTTP::Cookies::add_cookie_header: Checking .4.4.4 for cookies
HTTP::Cookies::add_cookie_header: Checking 4.4.4 for cookies
HTTP::Cookies::add_cookie_header: Checking .4.4 for cookies
HTTP::Cookies::add_cookie_header: Checking 4.4 for cookies
HTTP::Cookies::add_cookie_header: Checking .4 for cookies
LWP::UserAgent::send_request: GET http://10.4.4.4:8080/admin
LWP::Protocol::http::request: ()
HTTP::Cookies::extract_cookies: Set cookie JSESSIONID => blahblahblah
LWP::UserAgent::request: Simple response: OK
WWW::Mechanize=HASH(0x9429e0)->status()
WWW::Mechanize=HASH(0x9429e0)->content()
[download]

Is there something I am doing wrong in the first code example? The web server I am going against is JBOSS v4.2.2.
Thank you for your input. /rob

Comment on Problems with WWW::Mechanize Select or Download Code

Replies are listed 'Best First'.
Re: Problems with WWW::Mechanize by afresh1 (Hermit) on Oct 21, 2008 at 17:27 UTC
I took your first example, changed `$netloc`, `$realm`, `$username` and `$password` and it worked in my tests. One thing I do see is that if you manually set the Authorization header, as in your second example, it will send that header, and doesn't actually use the `$realm` set with `$agent->credentials()`, it works even with that line commented out. My guess with the available information would be that `$realm` is not correct. Perhaps you could try it with the two argument form of credentials? In my testing, an incorrect realm did not give me four attempts though, so that is a bit odd. l8rZ, -- andrew	[reply] [d/l] [select]
Re: Problems with WWW::Mechanize by Anonymous Monk on Oct 21, 2008 at 23:51 UTC
Maybe you need to upgrade lwp/mechanize/base64 to the latest `#!/usr/bin/perl -- use strict; use warnings; use WWW::Mechanize; use MIME::Base64; print " WWW::Mechanize $WWW::Mechanize::VERSION MIME::Base64 $MIME::Base64::VERSION LWP $LWP::VERSION "; __END__ WWW::Mechanize 1.34 MIME::Base64 3.05 LWP 5.814` [download]	[reply] [d/l]