in reply to Need help with HTTP::Proxy

As shmem said, HTTP::Proxy is probably not what you are seeking; leira's article "Web Testing with HTTP::Recorder" contains an excellent example of how you might want to use HTTP::Proxy in conjunction with WWW::Mechanize.

I think what you really want to know is "How do I use WWW::Mechanize through a proxy server?"

HTH,

planetscape

Replies are listed 'Best First'.
Re^2: Need help with HTTP::Proxy
by sunslinger (Initiate) on Jul 05, 2006 at 09:50 UTC
    Hi planetscape, I think you're spot on with my dilemma. Sorry if I wasn't clear enough as I'm really, really new with this. I tried the WWW::Mechanize via $mech->proxy(etc.) and compilation is ok. Then when I try to load the page $mech->get($url); I get this run-time(?) error:- Error GETing http://home/intranet/soemthing.htm: Proxy Authentication Required. The ISA server requires authorization to fulfill the request. Access to the Web Proxy service is denied. > at mech.pl at line 27 So basically I'm trying to get WWW::Mechanize to submit a form (I think) but the html page is behind my office's proxy and it prompts me for authentication. Please help!

        Hi planetscape,
        I've been searching through the perlmonks website and tried some of their recommendations but still could not seem to work. Maybe I should start from the beginning and hope this helps:

        1. Everytime i start my office desktop, I am required to logon to the NT domain in order to gain access to the Intranet. On Windows IE, I am required to set the company's proxy address and port number in order to get out to the Internet.

        2. What I would like to do is to run a perl script that is able to access an Intranet web page where I can then hopefully proceed to fill in some fields in form.

        Here's a cut and paste of my code:

        use WWW::Mechanize; use LWP::UserAgent; use HTTP::Proxy; use HTTP::Headers; use strict; my $realm = "Connecting to proxyname"; my $username= "\Domain\username"; my $password= "password"; my $url = "http://home/intranet/home/someurl.htm"; my $mech = WWW::Mechanize->new( autocheck => 1); my $proxy_url = "http://proxyname:80/"; my $proxy = HTTP::Proxy->new(); open (PROXYLOG, ">proxylog.txt") || die "Can't open proxylog.txt: $!" +; my $ua; { $ua = LWP::UserAgent->new; print PROXYLOG &patchdesc(1324710); } sub patchdesc { my $bugno = $_[0]; my $request = HTTP::Request->new( GET => $url . $bugno); # $request->header( 'Enter Network Password' => 'text/html' ); $ua->proxy( http => $proxy_url ); $request->proxy_authorization_basic($username,$password); return $ua->request($request)->as_string; } #my $http_user_agent = LWP::UserAgent->new; #$http_user_agent->agent( $user_agent ); #$http_user_agent->credentials( # $url, # $realm2, # $username => $password #); #my $response = $http_user_agent->get( $url ); #die "Error: ", $response->header($realm) || 'Error accessing', # "\n", $response->status_line, "\n at $url\n Aborting" # unless $response->is_success;

        And here's the printout from the Proxylog.Log file after running the script:

        HTTP/1.1 502 Proxy Error ( The ISA Server denies the specified Uniform + Resource Locator (URL). ) Cache-Control: no-cache Connection: close Pragma: no-cache Via: 1.1 MOCPROXY04 Content-Length: 2315 Content-Type: text/html Content-Type: text-html; charset=Windows-1252 Client-Date: Mon, 10 Jul 2006 07:08:16 GMT Client-Peer: 10.21.90.145:80 Client-Response-Num: 1 Proxy-Connection: close Title: The page cannot be displayed X-Meta-GENERATOR: MSHTML 5.50.4522.1800 X-Meta-ROBOTS: NOINDEX <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> <HTML dir=ltr><HEAD><TITLE>The page cannot be displayed</TITLE>

        ...something here which i think not important..

        <H2 style="FONT: 8pt/11pt verdana; COLOR: #000000">HTTP 502 Prox +y Error - The ISA Server denies the specified Uniform Resource Locato +r (URL). (12202)<BR>Internet Security and Acceleration Server</H2> <HR color=#c0c0c0 noShade> <P>Technical Information (for support personnel)</P> <UL> <LI>Background:<BR>The gateway could not retrieve the requeste +d page.<P></P></LI> <LI>ISA Server: mocproxy04.someproxy.local<BR> Via: <BR><BR>Time: 7/10/2006 7:08:15 AM GMT </LI></UL></FONT></TD></TR></TBODY></TABLE></BODY></HTML>

        Please help... been tearing up my hair over this trying to figure it out by myself...
        sunslinger

        20060710 Janitored by Corion: Added formatting, code tags, as per Writeup Formatting Tips