Preceptor has asked for the wisdom of the Perl Monks concerning the following question:

I'm currently experimenting with LWP, and have run into something of a major stumbling block - specifically, we have webwasher and a proxy on our outbound connections.
If I try a simple 'get' request using LWP, I don't get a failure, I get a page that informs me that (the URL) "has been blocked by Webwasher because you have not been authorized and authorization is required."
Now, what I'm having trouble with now, is that I can't figure out how to go to the state of 'authorized' - I've tried configuring a proxy, proxy user, proxy pass. This doesn't work. 
 use LWP::UserAgent;
 use HTTP::Request::Common;
 my $url = 'http://www.bbc.co.uk/';

 # Set up the ntlm client and then the base64 encoded ntlm handshake m
+essage
 my $ua = new LWP::UserAgent(keep_alive=>1);
 #$ua -> proxy ('http', "http://my_domain\\myuser:mypass@myproxy:8080"
+ );
 $ua -> proxy ('http', "http://myproxy:8080" );
 $ua->credentials('www.bbc.co.uk:80', '', "mydomain\\myuser", 'mypass'
+);
 $ua->credentials('myproxy:8080', '', "mydomain\\myuser", 'mypass');

 $request = GET $url;
 print "--Performing request now...-----------\n";
 $response = $ua->request($request);
 print "--Done with request-------------------\n";

 if ($response->is_success) {print "It worked!->" . $response->code . 
+"\n"}
 else {print "It didn't work!->" . $response->code . "\n"}

[download]
This doesn't seem to help - I still get a '407 authorization required' although I can still request web pages through IE or Firefox. Firefox does request a username and password, and the best I can come up with is that I need to use NTLM authentication - but I can't seem to figure out how, and was wondering if anyone could shed some further light? (for bonus points, give me a hint as to how I get the CPAN module working again :))
I'm using Activestate Perl, on W2K3 which makes things a little more interesting.
That code snippet is right out of the LWP::Authen::NTLM module, although I couldn't quite figure out if I need to be setting credentials to the proxy server, or to the target site. Proxy is autoconfigured normally, but I've dug out the actual IP/Port to be using manually. I'm pretty sure this _used_ to work, but think they've upgraded the proxy software.

Replies are listed 'Best First'.
Re: Proxy authentication using NTLM?
by keszler (Priest) on Dec 18, 2009 at 13:10 UTC
    Personally, I like to see exactly what's happening.
    • Run Wirehark
    • configure to capture only traffic to proxy server/port
    • capture IE/Firefox request/response
    • capture Perl request/response

    Comparison between the two will hopefully tell you something.

[reply]

      Try setting your User-Agent string to the same thing Firefox sends.

      As keszler notes, you need to capture what works. I like to use the paros proxy for capturing http traffic.

      Update.... Oh Dang my brain is just catching up with my fingers. How can one use a proxy to debug another proxy? Paros can forward traffic to a second proxy but I don't know if the authentication will work then.

[reply]
Re: Proxy authentication using NTLM?
by Khen1950fx (Canon) on Dec 18, 2009 at 19:56 UTC
    This worked for me:

    #!/usr/local/bin/perl

use strict;
use warnings;
use LWP::UserAgent::Determined;
use HTTP::Request::Common;
use Authen::NTLM;
ntlmv2(1);
 
my $url = 'http://www.bbc.co.uk/';

my $ua = LWP::UserAgent::Determined->new(keep_alive=>1);
$ua->credentials('www.bbc.co.uk:80', '', 'mydomain\\myuser', 'mypass')
+;
 
my $request = GET $url;
print "--Performing request now...-----------\n";
my $response = $ua->request($request);
print "--Done with request-------------------\n";

if ($response->is_success) {print "It worked!->" . $response->code . "
+\n"}
else {print "It didn't work!->" . $response->code . "\n"}

my $contents = $response->content();
print $response->status_line(), "\n";
print $response->headers()->as_string(), "\n";

    [download]
    Update: I couldn't get it to work correctly without setting $ua->env_proxy. Edited script to reflect that.
    Update2: Last fix. Added ntlmv2 and decided to stick with the credentials method. Eliminated $ua->env_proxy; added my $contents
[reply]
[d/l]

Back to Seekers of Perl Wisdom