in reply to Re^2: Need help with HTTP::Proxy
in thread Need help with HTTP::Proxy

Super Searching for "Proxy Authentication Required" yields the following:

I would probably start there and see if any of those suggestions help.

My boilerplate advice for dealing with WWW::Mechanize issues is:

I would try using a module such as HTTP::Recorder or WWW::Mechanize::Shell to record a successful manual form submission. The output of HTTP::Recorder, for instance, can be "dropped" right into your WWW::Mechanize scripts (leira's article that I mentioned above tells you how).

Another important tool for finding out what is really happening behind the scenes between server and browser is a protocol analyzer such as Ethereal.

If none of those suggestions helps, try posting your entire script as a reply here - censoring any sensitive details such as username/password, of course... And let us know more about your environment...

HTH,

planetscape

Replies are listed 'Best First'.
Need help with LWP::UserAgent and accessing through to proxy
by Anonymous Monk on Jul 10, 2006 at 07:18 UTC

    Hi planetscape,
    I've been searching through the perlmonks website and tried some of their recommendations but still could not seem to work. Maybe I should start from the beginning and hope this helps:

    1. Everytime i start my office desktop, I am required to logon to the NT domain in order to gain access to the Intranet. On Windows IE, I am required to set the company's proxy address and port number in order to get out to the Internet.

    2. What I would like to do is to run a perl script that is able to access an Intranet web page where I can then hopefully proceed to fill in some fields in form.

    Here's a cut and paste of my code: 

    use WWW::Mechanize;
use LWP::UserAgent;
use HTTP::Proxy;
use HTTP::Headers;
use strict;

my $realm  = "Connecting to proxyname";
my $username= "\Domain\username";
my $password= "password";
my $url      = "http://home/intranet/home/someurl.htm";
my $mech     = WWW::Mechanize->new( autocheck => 1);
my $proxy_url = "http://proxyname:80/";
my $proxy    = HTTP::Proxy->new();

open (PROXYLOG, ">proxylog.txt") || die "Can't open proxylog.txt: $!" 
+;

my $ua;
{
    $ua = LWP::UserAgent->new;
    print PROXYLOG &patchdesc(1324710);
}

sub patchdesc {
    my $bugno    = $_[0];
    my $request = HTTP::Request->new( GET => $url . $bugno);
#    $request->header( 'Enter Network Password' => 'text/html' );
    $ua->proxy( http => $proxy_url );
    $request->proxy_authorization_basic($username,$password);

    return $ua->request($request)->as_string;
}
    

#my $http_user_agent = LWP::UserAgent->new;
#$http_user_agent->agent( $user_agent );
#$http_user_agent->credentials(
#                    $url,
#                    $realm2,
#                    $username => $password
#);

#my $response = $http_user_agent->get( $url );

#die "Error: ", $response->header($realm) || 'Error accessing',
#    "\n", $response->status_line, "\n at $url\n Aborting"
#    unless $response->is_success;

    [download]

    And here's the printout from the Proxylog.Log file after running the script:

    HTTP/1.1 502 Proxy Error ( The ISA Server denies the specified Uniform
+ Resource Locator (URL).  )
Cache-Control: no-cache
Connection: close
Pragma: no-cache
Via: 1.1 MOCPROXY04
Content-Length: 2315  
Content-Type: text/html
Content-Type: text-html; charset=Windows-1252
Client-Date: Mon, 10 Jul 2006 07:08:16 GMT
Client-Peer: 10.21.90.145:80
Client-Response-Num: 1
Proxy-Connection: close
Title: The page cannot be displayed
X-Meta-GENERATOR: MSHTML 5.50.4522.1800
X-Meta-ROBOTS: NOINDEX

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">

<HTML dir=ltr><HEAD><TITLE>The page cannot be displayed</TITLE>

    [download]

    ...something here which i think not important.. 

          <H2 style="FONT: 8pt/11pt verdana; COLOR: #000000">HTTP 502 Prox
+y Error - The ISA Server denies the specified Uniform Resource Locato
+r (URL). (12202)<BR>Internet Security and Acceleration Server</H2>

      <HR color=#c0c0c0 noShade>



      <P>Technical Information (for support personnel)</P>

      <UL>

        <LI>Background:<BR>The gateway could not retrieve the requeste
+d page.<P></P></LI>

        <LI>ISA Server: mocproxy04.someproxy.local<BR>

        Via: <BR><BR>Time: 7/10/2006 7:08:15 AM GMT

        </LI></UL></FONT></TD></TR></TBODY></TABLE></BODY></HTML>

    [download]

    Please help... been tearing up my hair over this trying to figure it out by myself...
    sunslinger

    20060710 Janitored by Corion: Added formatting, code tags, as per Writeup Formatting Tips

[reply]
[d/l]
[select]
[reply]

      Sorry, let me write that out in a proper format:

      Hi planetscape, I've been searching through the perlmonks website and tried some of their recommendations but still could not seem to work. Maybe I should start from the beginning and hope this helps:


      1. Everytime i start my office desktop, I am required to logon to the NT domain in order to gain access to the Intranet. On Windows IE, I am required to set the company's proxy address and port number in order to get out to the Internet.
      2. What I would like to do is to run a perl script that is able to access an Intranet web page where I can then hopefully proceed to fill in some fields in form.
      Here's a cut and paste of my code: 
      use WWW::Mechanize; 
use LWP::UserAgent; 
use HTTP::Headers; 
use strict; 
my $realm = "Connecting to proxyname";
my $username= "\Domain\username";
 my $password= "password";
 my $url = "http://home/intranet/home/someurl.htm";
 my $mech = WWW::Mechanize->new( autocheck => 1);
 my $proxy_url = "http://proxyname:80/";
 my $proxy = HTTP::Proxy->new();

 open (PROXYLOG, ">proxylog.txt") || die "Can't open proxylog.txt: $!"
+ ;

 my $ua; 
{ $ua = LWP::UserAgent->new; 
print PROXYLOG &patchdesc(1324710); }

 sub patchdesc { 
        my $bugno = $_[0]; 
        my $request = HTTP::Request->new( GET => $url . $bugno); 
        $ua->proxy( http => $proxy_url ); 
        $request->proxy_authorization_basic($username,$password); 
        return $ua->request($request)->as_string; } 

#my $http_user_agent = LWP::UserAgent->new; #$http_user_agent->agent( 
+$user_agent ); #$http_user_agent->credentials( 
    # $url, 
    # $realm2, 
    # $username => $password 
#); 

#my $response = $http_user_agent->get( $url ); 

#die "Error: ", $response->header($realm) || 'Error accessing', 
# "\n", $response->status_line, "\n at $url\n Aborting" # unless $resp
+onse->is_success;

      [download]


      And here's the printout from the Proxylog.Log file after running the script: 
      HTTP/1.1 502 Proxy Error ( The ISA Server denies the specified Uniform
+ Resource Locator (URL). ) 
Cache-Control: no-cache 
Connection: close 
Pragma: no-cache 
Via: 1.1 MOCPROXY04
Content-Length: 2315 
Content-Type: text/html 
Content-Type: text-html; charset=Windows-1252 
Client-Date: Mon, 10 Jul 2006 07:08:16 GMT 
Client-Peer: 10.21.90.145:80 
Client-Response-Num: 1 
Proxy-Connection: close 
Title: The page cannot be displayed 
X-Meta-GENERATOR: MSHTML 5.50.4522.1800 
X-Meta-ROBOTS: NOINDEX 

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> <HTML d
+ir=ltr><HEAD><TITLE>The page cannot be displayed</TITLE> ...something
+ here which i think not important.. 


      <H2 style="FONT: 8pt/11pt verdana; COLOR: #000000">HTTP 502 Prox
+y Error - The ISA Server denies the specified Uniform Resource Locato
+r (URL). (12202)Internet Security and Acceleration Server</H2>

      <HR color=#c0c0c0 noShade>



      <P>Technical Information (for support personnel)</P>

      <UL>

        <LI>Background:The gateway could not retrieve the requested pa
+ge.<P></P></LI>

        <LI>ISA Server: mocproxy04.m1.local

        Via: Time: 7/10/2006 7:08:15 AM GMT

        </LI></UL></FONT></TD></TR></TBODY></TABLE></BODY></HTML>

      [download]

      20060710 Janitored by Corion, again: Added formatting, code tags, as per Writeup Formatting Tips

[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom