in reply to HTTPS connection with LWP and self-signed certificate

I don't know what you did and how your certificate looks like, but this is what I did and which works with the current version of IO::Socket::SSL. 
# create a self-signed certificate
perl -MIO::Socket::SSL::Utils -e '
   my ($c,$k) = CERT_create(CA => 1, subject => { CN => q[foo.bar]}); 
   print PEM_cert2string($c).PEM_key2string($k)' \
   > self-signed.pem

# start a server with this certificate (default port 4433)
openssl s_server -cert self-signed.pem -key self-signed.pem &

# connect to this server
perl -Ilib -MIO::Socket::SSL=debug4 -e '
   my $cl = IO::Socket::SSL->new(
      PeerAddr => q[127.0.0.1:4433], 
      SSL_hostname => q[foo.bar], 
      SSL_ca_file => q[self-signed.pem]
   ) or die "$SSL_ERROR,$!";
   warn "SUCCESS cipher=".$cl->get_cipher();'

[download]
This gives me (among all the debug output) "SUCCESS cipher=ECDHE-RSA-AES128-SHA256" which means, that the SSL connection was successfully established. You might check with your certificate to make sure, that the problem is not the certificate itself.

Replies are listed 'Best First'.
Re^2: HTTPS connection with LWP and self-signed certificate
by Anonymous Monk on Jan 07, 2015 at 23:10 UTC

    I upgraded my IO::Socket::SSL to the latest version, it took a while since I also had to upgrade a bunch of other stuff to get it to work, but at least now I have all the utilities.

    I ran your code to generate a certificate, setup the server and connect to it, that all worked. Next I ran the server on the host which I'm trying to connect to and modified the ip for the client, still worked. Then I configured apache to use that exact certificate and and again it worked!

    Convinced that all my troubles were over I tried to execute my script from the original post to see if it would also work with the new certificate but...

    DEBUG: .../IO/Socket/SSL.pm:2555: new ctx 34454560
DEBUG: .../IO/Socket/SSL.pm:539: socket not yet connected
DEBUG: .../IO/Socket/SSL.pm:541: socket connected
DEBUG: .../IO/Socket/SSL.pm:563: ssl handshake not started
DEBUG: .../IO/Socket/SSL.pm:599: not using SNI because hostname is unk
+nown
DEBUG: .../IO/Socket/SSL.pm:631: request OCSP stapling
DEBUG: .../IO/Socket/SSL.pm:650: set socket to non-blocking to enforce
+ timeout=180
DEBUG: .../IO/Socket/SSL.pm:663: Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:673: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:683: waiting for fd to become ready: SSL w
+ants a read first
DEBUG: .../IO/Socket/SSL.pm:703: socket ready, retrying connect
DEBUG: .../IO/Socket/SSL.pm:2458: did not get stapled OCSP response
DEBUG: .../IO/Socket/SSL.pm:2411: ok=1 cert=34885312
DEBUG: .../IO/Socket/SSL.pm:1559: scheme=www cert=34885312
DEBUG: .../IO/Socket/SSL.pm:1569: identity=192.168.100.222 cn=________
+_____ alt=
DEBUG: .../IO/Socket/SSL.pm:1769: hostname verification failed
DEBUG: .../IO/Socket/SSL.pm:663: Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:1780: SSL connect attempt failed

DEBUG: .../IO/Socket/SSL.pm:1785: SSL connect attempt failed error:140
+90086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify fai
+led
DEBUG: .../IO/Socket/SSL.pm:669: fatal SSL error: SSL connect attempt 
+failed error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certif
+icate verify failed
DEBUG: .../IO/Socket/SSL.pm:1769: IO::Socket::IP configuration failed
DEBUG: .../IO/Socket/SSL.pm:2588: free ctx 34454560 open=34454560
DEBUG: .../IO/Socket/SSL.pm:2593: free ctx 34454560 callback
DEBUG: .../IO/Socket/SSL.pm:2600: OK free ctx 34454560
500 Can't connect to 192.168.100.222:4433 (certificate verify failed)
Content-Type: text/plain
Client-Date: Wed, 07 Jan 2015 23:00:51 GMT
Client-Warning: Internal response

Can't connect to 192.168.100.222:4433 (certificate verify failed)

SSL connect attempt failed error:14090086:SSL routines:SSL3_GET_SERVER
+_CERTIFICATE:certificate verify failed at /usr/local/share/perl/5.14.
+2/LWP/Protocol/http.pm line 49

    [download]
    On the serverside openssl s_server said: 
    Using default temp DH parameters
Using default temp ECDH parameters
ACCEPT
bad gethostbyaddr
ERROR
140707196729000:error:14094416:SSL routines:SSL3_READ_BYTES:sslv3 aler
+t certificate unknown:s3_pkt.c:1256:SSL alert number 46
shutting down SSL
CONNECTION CLOSED
ACCEPT

    [download]

    trying to connect to apache with the new cert also failed. This leaves me to think that there is something wrong with (the way I'm using) LWP::UserAgent.

    Is it possible to handle the ssl connection with IO::Socket::SSL directly but still have all the LWP::UserAgent functionality for my interaction with the server after the connection is established?

[reply]
[d/l]
[select]
      Is it possible to handle the ssl connection with IO::Socket::SSL directly but still have all the LWP::UserAgent functionality for my interaction with the server after the connection is established?
      No, LWP wants to have full control over the socket and does not allow to create a connection using an already established socket. Apart from that, the following code works for me with LWP::UserAgent 6.05 and LWP::Protocol::https 6.04 (with the modification done in Ubuntu 14.04): 
      use strict;
use warnings;
use LWP::UserAgent;

my $ua = LWP::UserAgent->new;
$ua->ssl_opts( SSL_ca_file => 'self-signed.pem');
# set verification name explicitly for this test because the
# URL does not contain the correct name
$ua->ssl_opts( SSL_verfifycn_name => 'foo.bar');
my $res = $ua->get('https://127.0.0.1:4433');
print $res->as_string;

      [download]
      In this case s_server is started with -WWW, that is: 
      openssl s_server -cert self-signed.pem -key self-signed.pem -WWW

      [download]
      Which leaves the question about the version of LWP::UserAgent and LWP::Protocol::https you are using.
[reply]
[d/l]
[select]

        adding the SSL_verifycn_name option fixed it!

        also, from your "# URL does not contain the correct name" comment I got that LWP tries to automatically derive this from the url if it contains a domain name and not just the ip.

        To test this I swapped the ip for the domain name (which I originally did not do because it does not resolve to the internal ip and hence does not use the direct route). This also allowed it to work, even without directly specifying the cn via ssl_opts.

        I'd like to thank you and all the other monks who helped to resolve this problem!

        It is annoying to see how so much time can go into finding that you just need one extra option for your connection...

[reply]

        Is it possible to handle the ssl connection with IO::Socket::SSL directly but still have all the LWP::UserAgent functionality for my interaction with the server after the connection is established?

        No, LWP wants to have full control over the socket and does not allow to create a connection using an already established socket.

        Well, yes it is possible -- see Re: LWP is there any way to get "real" outgoing headers? -- but it doesn't make much sense .... if there is a bug in LWP suite, fix the bug, not work around it

[reply]

In Section Seekers of Perl Wisdom