HTTPS requests

Yukio has asked for the wisdom of the Perl Monks concerning the following question:

Does anyone know a robust way to connect to a secure web server using HTTPS? I'm trying to port over some scripts that connect to web servers using the more conventional HTTP protocol. When I run the script, the response object comes back with a false is_success. I am presuming that I'm getting stopped by LWP::UserAgent, possibly because of an the (empty) proxy setting (I don't understand enough about the proxy parameter and unfortunately I don't have time at the moment to learn about it). Set on debug, I get the following information:

LWP::UserAgent::new: ()
LWP::UserAgent::request: ()
LWP::UserAgent::simple_request: GET https://www.asecureserver.com/
LWP::UserAgent::_need_proxy: Not proxied
LWP::UserAgent::request: Simple response: Not Implemented

And here's the simplified script:

#!\usr\bin\perl -w
use LWP;
use LWP::Debug '+';
require HTTP::Request;
use LWP::UserAgent;

use Carp;
use strict;

my ($url, $useragent, $request, $response, $response_text);

$url = 'https://www.asecureserver.com/'; # not a real url
$useragent = LWP::UserAgent->new;
$request = HTTP::Request->new('GET', $url);

$response = $useragent->request($request);

if ($response->is_success) {
   print $response->content;
}
else {
	 print 'Failed';
}

Any and all help is greatly appreciated.

Comment on HTTPS requests

Replies are listed 'Best First'.
Re: HTTPS requests by cacharbe (Curate) on Sep 21, 2001 at 21:25 UTC
You'll want to read lwpcook on HTTPS, but just in case, here's the section I'm speaking of: ------ URLs with https scheme are accessed in exactly the same way as with http scheme, provided that an SSL interface module for LWP has been properly installed (see the README.SSL file found in the libwww-perl distribution for more details). If no SSL interface is installed for LWP to use, then you will get ``501 Protocol scheme 'https' is not supported'' errors when accessing such URLs. Here's an example of fetching and printing a WWW page using SSL: `use LWP::UserAgent; my $ua = LWP::UserAgent->new; my $req = HTTP::Request->new(GET => 'https://www.helsinki.fi/'); my $res = $ua->request($req); if ($res->is_success) { print $res->as_string; } else { print "Failed: ", $res->status_line, "\n"; }` [download] ---- Hope that helps. C-.	[reply] [d/l]
Re: HTTPS requests by blakem (Monsignor) on Sep 21, 2001 at 21:27 UTC
You can use LWP for this, though you'll need a separate crypto module installed. I think you can use either Crypt::SSLeay or IO::Socket::SSL. See the README.SSL file in the libwww-perl distribution for further information. -Blake	[reply]
Re: HTTPS requests by shotgunefx (Parson) on Sep 22, 2001 at 01:25 UTC
Just a thought, don't know what you are running but there seems to be a problem with LWP and https on (certain) ultrasparc/linux combo. All the tests when installing it work. When you request a secure url, it actually does go and get it. (Called another of my servers to verify) is_success always fails. So do all of the get methods for Simple and UserAgent. I submitted a bug report on SourceForge but never heard anything. I fiddled with it for months and just gave up and moved the https stuff to a secure box. SSL is installed and working correctly on the box. It's only LWP and IO:Socket modules that won't work with it. Somewhere in the protocol negotiation, it fails. Actually, LWP::Simple::get doesn't work either. But getstore() does. -Lee "To be civilized is to deny one's nature."	[reply]