simon.proctor has asked for the wisdom of the Perl Monks concerning the following question:

I've been running a news fetcher at work sucessfully for some time, until WEBSweeper was installed. The problem now is that the standard method of authenticating (as shown in the LWP cookbook doesn't work. For example: 
my $ua = LWP::UserAgent->new;

# Proxy location
$ua->proxy(['http', 'ftp'] => 'proxy-url');

# What we want - hardcoded
my $req = HTTP::Request->new('GET',"target-url");

# Authenticate against the proxy - hardcoded
$req->proxy_authorization_basic("domain/user", "password");

[download]

Sadly I now get a 407 and, before you ask, the domain, username and password are upto date :). In fact this worked without a hitch on MSProxy v2.0. Now I've done some searching and NTLM authentication appears (at a glance) to be what I need to do but frankly I have NO IDEA what that is or how to do it.

Even more frustrating is that the following Java code never broke: 
URL url = new URL("target");
String authString = "domain/user:password";      
String auth = "Basic " + new sun.misc.BASE64Encoder().encode(authStrin
+g.getBytes());
URLConnection conn = url.openConnection();
            conn.setRequestProperty("Proxy-Authorization", auth);

[download]
But I can't think of how to emulate that in Perl, or even if I should. I've also tried the other authentication methods that I could find (as well as mucking about with the headers) but that was just a case of shotgun debugging that I shouldn't really admit to :)

Can anyone help?

Replies are listed 'Best First'.
Re: WEBSweeper Proxy Authentication
by PodMaster (Abbot) on Jul 17, 2002 at 10:36 UTC
    I have no idea what you're talking about until we get into the
    "But I can't think of how to emulate that in Perl ..."

    It's pretty easy, you're using all the right tools, but you're missing one (not really) so check it out. 

    use MIME::Base64;
use HTTP::Request;
use LWP::UserAgent;


sub IamGET {
    my ($url, $domain, $user, $pass) = @_;

    my $REQ = HTTP::Request->new(GET => $url);

    my $authString  = encode_base64("$domain/$user:$pass");
    $REQ->push_header("Proxy-Authorization", "Basic $authString");

    my $UA = LWP::UserAgent->new;
    $UA->agent("Mozilla/6.9(${^O};retmaspod)");
    my $RES = $UA->request($REQ);

    if($RES->is_success()) {
        return $RES->content;
    } else {
        $! = " Error (".scalar(localtime())."):". $RES->status_line . 
+"\n";
        return undef;
    }
}

    [download]
    update:
    I'm not quite so sure that it's supposed to work. It does in fact emulate what the java code does to the best of my knowledge.

    Check out this for ideas on where you stand.

    LWP::UserAgent only supports basic http authentication. If you still need to use NTLM, check cpan. Since it is an HTTP protocol, you can probably find an RFC somewhere, and push whatever extra headers it reqires, which you can do using the method I demonstrated above. You might wanna also check out http://squid.sourceforge.net/ntlm/

[reply]
[d/l]
      Thanks for trying but it doesn't work. Your code does not include a proxy line so I get: 
      Error (Wed Jul 17 11:55:58 2002):500 Can't connect to www.yahoo.co.uk:
+80 (Timeout)

      [download]
      However, if I add the proxy line I then get: 
      # Added
$UA->proxy(['http', 'ftp'] => 'http://proxyurl');

# Error
Error (Wed Jul 17 11:55:05 2002):407 Proxy authentication required

      [download]
      But thanks anyway.
[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom