LWP SSL Question

packetstormer has asked for the wisdom of the Perl Monks concerning the following question:

Howd'y Monks

I have a wierd problem where if I telnet to an https host on the port (443 or 8443) and/or use curl to post to the same url all works OK. However, if I use LWP::UserAgent I get an error 500 - cannot connect to $site.

I am using some very simple code:

#!/usr/bin/perl
use strict;
use warnings;
use LWP::UserAgent;
use Data::Dumper;

my $host = 'localhost';
my $port = '8443';
my $ua = LWP::UserAgent->new(ssl_opts => { verify_hostname => 0});
$ua->agent("Mozilla/29.0");
my $req = HTTP::Request->new( GET => "https://$host:$port" );
$req->header( 'Accept' => 'text/html' );
my $res = $ua->request($req);
print Dumper $res;
[download]

This code returns an error 500
However, curl or a simple telnet to the same host:port gets a connection (although obviously encrytped in the telnet connection)
Would anyone have any ideas!?

EDIT: Added my declaration per strict etc....

Comment on LWP SSL Question Download Code

Replies are listed 'Best First'.
Re: LWP SSL Question by zentara (Cardinal) on Jun 06, 2014 at 18:22 UTC
Without LWP-Protocol-https installed, I get a message to install it when running your code. After installing it, I get a "connection refused" message. Are you sure your port is 8443? and not 443? Your code works fine if I change the target: `my $req = HTTP::Request->new( GET => "https://signin.ebay.com" );` [download] I'm not really a human, but I play one on earth. Old Perl Programmer Haiku ................... flash japh	[reply] [d/l]
Re: LWP SSL Question by hippo (Archbishop) on Jun 06, 2014 at 16:36 UTC
It's worse than that - your code doesn't even compile. Global symbol "$ua" requires explicit package name at 1089058.pl line +9. Global symbol "$ua" requires explicit package name at 1089058.pl line +10. Global symbol "$req" requires explicit package name at 1089058.pl line + 11. Global symbol "$req" requires explicit package name at 1089058.pl line + 12. Global symbol "$res" requires explicit package name at 1089058.pl line + 13. Global symbol "$ua" requires explicit package name at 1089058.pl line +13. Global symbol "$req" requires explicit package name at 1089058.pl line + 13. Global symbol "$res" requires explicit package name at 1089058.pl line + 14. Execution of 1089058.pl aborted due to compilation errors. [download] But, once I fixed those, it works fine for me using real values for `$host` and `$port`. As taint says, check the error log on your local server to see where the problem lies. Update: Parent post has been altered and the source now compiles as-is.	[reply] [d/l] [select]
Re: LWP SSL Question by MidLifeXis (Monsignor) on Jun 06, 2014 at 17:20 UTC
Do you have LWP::Protocol::https installed? It was split from LWP a bit ago, and this sounds like the behavior I remember. --MidLifeXis	[reply]
Re^2: LWP SSL Question by packetstormer (Monk) on Jun 06, 2014 at 17:42 UTC
LWP::Protocol::https is installed, yep. Pity, thought I might be on to something! I can't get to any logs on the remote server to see what is happening which is also a pity!	[reply]
Re^3: LWP SSL Question by MidLifeXis (Monsignor) on Jun 06, 2014 at 17:43 UTC
Correctly configured, proper root certificates available, and so on? --MidLifeXis	[reply]
Re^4: LWP SSL Question by jellisii2 (Hermit) on Jun 06, 2014 at 17:50 UTC
Re: LWP SSL Question by noxxi (Pilgrim) on Jun 07, 2014 at 11:29 UTC
From your code you connect to localhost 8443. I doubt, that you have a valid certificate there, e.g. one which can be verified against a trusted CA provided by the Mozilla::CA CA store. Thus it will fail to connect because the certificate can not be validated. You can check this by adding SSL debugging with: perl -MIO::Socket::SSL=debug10 yourcode.pl Setting verify_hostname to 0 will not help, because this concerns only the validation of the hostname inside the certificate and not the validation of the certificate against the list of trusted CAs (at least it should, in some versions of LWP it actually disables all verification which is wrong). If you really need to disable verification you should set SSL_verify_mode accordingly (see documentation of IO::Socket::SSL), but in this case you should ask yourself why you use SSL at all.	[reply]
Re^2: LWP SSL Question by packetstormer (Monk) on Jun 07, 2014 at 18:10 UTC
So, the 'debug10' trick helped a lot and from the output I was able to narrow down the problem to the cipher being used. Adding `SSL_cipher_list => "RC4-SHA"` to my ssl_opts resolved the problem and got me talking to the SSL server Thanks for the replies, they certainly helped!	[reply] [d/l]
Re^3: LWP SSL Question by noxxi (Pilgrim) on Jun 07, 2014 at 19:30 UTC
This is strange, because RC4-SHA is in the default cipher set used by IO::Socket::SSL on the client side (but only as the last choice). This must be a very strange SSL server you have there, could you share some details about it? (I'm the maintainer of IO::Socket::SSL and always on the search for unusual SSL issues.)	[reply]
Re^4: LWP SSL Question by packetstormer (Monk) on Jun 08, 2014 at 09:56 UTC
Re: LWP SSL Question by taint (Chaplain) on Jun 06, 2014 at 16:18 UTC
Greetings, packetstormer. I don't suppose the host you're attempting to connect to has an error log, or any logging facility, that emits (error) messages, that might elude to the (HTTP)-500 response? :) Best wishes. --Chris Ąλɐp ʇɑəɹ⅁ ɐ əʌɐɥ puɐ ʻꜱdləɥ ꜱᴉɥʇ ədoH	[reply]