Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Dear Monks,

I am trying to make an HTTPS request across the proxy server. I am using LWP::UserAgent. I have downloaded and installed openssl and Crypt::SSleay. My operating system is Windows 2000. My HTTPS request is successful, but I do not get any content. The response->as_string method outputs:

#############################################
HTTP/1.0 200 OK
Content-Type: text/html
Client-Date: Fri, 13 Dec 2002 02:39:25 GMT
Client-Response-Num: 1
Refresh: 0; URL=https://www.ultimatix.net
<HTML></HTML>
############################################

This happens whatever I try to do. I have tried out almost every thing. I have even created ssl certificate and used the same. But nothing works. I do not get the content.

My code looks like this 

sub GetBaseURL{
my($lstrURL)=@_;
my $ua = LWP::UserAgent->new;
$ua->agent("MyApp/0.1");
$ua->proxy([qw(https http)], "http://proxy.compaq.com:8080");
$ENV{HTTPS_VERSION}='3';
#$ENV{HTTPS_CERT_FILE}="C:/openssl/private/CAcert.pem";
#$ENV{HTTPS_KEY_FILE}="C:/openssl/private/CAkey.pem";
$ENV{HTTPS_PKCS12_FILE}="C:/openssl/binu.p12";
$ENV{HTTPS_PKCS12_PASSWORD}='passcode';
#$ENV{HTTPS_PROXY}="http://proxy.compaq.com:8080";
$ua->timeout(1000);
my $request=HTTP::Request->new(POST=> $lstrURL);
my $response=$ua->request($request);
#my $response=$ua->get($lstrURL) or die "cannot fetch $lstrURL: $!";
if ($response->is_success){
    return $response->as_string;
}else{
    return $response->code;
}


}

[download]
Please bail me out. Suggest me a way to make HTTPS request across the proxy.

Regards,
Binu

Replies are listed 'Best First'.
Re: HTTPS Call across proxy server
by pg (Canon) on Dec 13, 2002 at 04:25 UTC
    You already got what you were supposed to get.

    You did: 
    my $request=HTTP::Request->new(POST=> $lstrURL);

    [download]
    which is a post request, and you got a 200 OK back, which meant that your request had been received, understood and processed by the server.

    As you seem to be surprised at this response, and also you said "I do not get the content", I assume what you actually wanted to do was a 'GET' instead of 'POST', then you should do something like: 
    my $request=HTTP::Request->new(GET => $lstrURL);

    [download]
    Update: Thanks to jjdraco for pointing out my typo.
[reply]
[d/l]
[select]
      As you seem to be surprised at this response, and also you said "I do not get the content", I assume what you actually wanted to do was a 'GET' instead of 'POST', then you should do something like: 
      my $request=HTTP::Request->new(POST=> $lstrURL);

      [download]
       I think you meant to say 
      my $request=HTTP::Request->new(GET=> $lstrURL);

      [download]
      or am I wrong on what I thought I read?

      jjdraco
      learning Perl one statement at a time.
[reply]
[d/l]
[select]
      Dear jjdraco,
      Thanks a lot for the prompt response. I have used
      my $request=HTTP::Request->new(GET => $lstrURL);
      but the result is the same. I want to get the content between the <HTML></HTML> tags. I am unable to get that.
      Regards,
      Binu
[reply]
[d/l]
Re: HTTPS Call across proxy server
by hossman (Prior) on Dec 13, 2002 at 08:42 UTC

    Just to be sure there's no confusion, LWP isn't creating a that response for you out of thin air. that's working. If you didn't have the SSL stuff working you'd get an error when you tried this.

    Even if you have any doubts about the 200 status code, the "<HTML></HTML>" should be a dead give away.

[reply]
[d/l]
      Hi Hossman,

      I know that the request was successful that is why I got status code 200. I want to get the content so that I can parse the same. I will show the alternative to this one that I wrote using Win32::OLE. This uses msxml2.serverxmlhttp.


      sub GetContent{
my($lstrURL)=@_;
$testURL=Variant(VT_BSTR,$lstrURL);
my $lobjXML=Win32::OLE->new('MSXML2.ServerXMLHTTP');
#my $lobjXML=Win32::OLE->new('WinHttp.WinHttpRequest.5.1') or die "can
+not create: $!";
$lobjXML->open("GET",$testURL,false);
#$lobjXML->SetClientCertificate("Enterprise Trust\\Local Computer");
$lobjXML->send;
#print $lobjXML->GetALLResponseHeaders;
if($lobjXML->status==200){
    #print $lobjXML->Status ,"\n";
    return $lobjXML->ResponseText;
}
else{
    return $lobjXML->status,"\n";
    #print $lobjXML->ResponseText;
    
}
undef $lobjXML;

}

      [download]

      This piece of code works fine and fetches content. But I cannot port this on a linux box. That is why I want the whole thing to work using LWP. My requirement is to parse the content which I get as response to HTTPS request. Thanks a lot for the response.
      Regards,
      Binu

[reply]
[d/l]

        I think you're missing my point.

        The fact that you are getting a response back, with a body, means that your client code is working. If you aren't getting the content you expect, that's probably not a problem with your client code. What you need to figure out, is why the server/proxy might be treating your LWP client differently from your Win32::OLE client.

        One posibility allready discussed is the GET vs. POST issue. Another is the proxy ... I don't know much about Win32::OLE, but I don't see anything in your second version that makes it look like it's using a proxy. There's also the issue of the User Agent, the server (or the proxy) could be responding differently based on the User Agent header ... or for that matter, any header.

[reply]

Back to Seekers of Perl Wisdom