How do I ensure client side certificates are used in SSL handshaking?

ted.byers has asked for the wisdom of the Perl Monks concerning the following question:

I have SSL working, mostly, when communicating with a server on my own LAN (not publicly accessable). My browsers can connect to it using HTTPS. But, as you're aware, the usual printenv.pl script will dispay the contents of any certificates, both client and server and when I use LWP's user agent, and provide the client certificate file name, and the directory it is in, the printenv.pl script produces output indicating that the client side certificate is empty.

Here is how the SSL virtual host is configured:

# Template for a VirtualHost with SSL
# Note: to use the template, rename it to /etc/apache2/vhost.d/yourvho
+st.conf.
# Files must have the .conf suffix to be loaded.
#
# See /usr/share/doc/packages/apache2/README.QUICKSTART for further hi
+nts
# about virtual hosts.

# NameVirtualHost statements should be added to /etc/apache2/listen.co
+nf.

#
# This is the Apache server configuration file providing SSL support.
# It contains the configuration directives to instruct the server how 
+to
# serve pages over an https connection. For detailing information abou
+t these 
# directives see http://httpd.apache.org/docs/2.2/mod/mod_ssl.html
#
# Do NOT simply read the instructions in here without understanding
# what they do.  They're here only as hints or reminders.  If you are 
+unsure
# consult the online docs. You have been warned.  
#

<IfDefine SSL>
<IfDefine !NOSSL>

##
## SSL Virtual Host Context
##

<VirtualHost _default_:443  >
    ServerAdmin r.ted.byers@gmail.com
    ServerName gremlin.site
    DocumentRoot /srv/www/vhosts/gremlin.site

    #  General setup for the virtual host
    #DocumentRoot "/srv/www/htdocs"
    #ServerName www.example.com:443
    #ServerAdmin webmaster@example.com
#    ErrorLog /var/log/apache2/error_log
#    TransferLog /var/log/apache2/access_log
    ErrorLog /var/log/apache2/gremlin.site-error_log
    CustomLog /var/log/apache2/gremlin.site-access_log combined

    #   SSL Engine Switch:
    #   Enable/Disable SSL for this virtual host.
    SSLEngine on

    #  SSL protocols
    #  Supporting TLS only is adequate nowadays
    SSLProtocol all -SSLv2

    #   SSL Cipher Suite:
    #   List the ciphers that the client is permitted to negotiate.
    #   See the mod_ssl documentation for a complete list.
    SSLCipherSuite HIGH:MEDIUM:!aNULL:!MD5

    SSLVerifyClient optional
    SSLVerifyDepth 10
    SSLOptions +ExportCertData +StdEnvVars

    #   Speed-optimized SSL Cipher configuration:
    #   If speed is your main concern (on busy HTTPS servers e.g.),
    #   you might want to force clients to specific, performance
    #   optimized ciphers. In this case, prepend those ciphers
    #   to the SSLCipherSuite list, and enable SSLHonorCipherOrder.
    #   Caveat: by giving precedence to RC4-SHA and AES128-SHA
    #   (as in the example below), most connections will no longer
    #   have perfect forward secrecy - if the server's key is
    #   compromised, captures of past or future traffic must be
    #   considered compromised, too.
    #SSLCipherSuite RC4-SHA:AES128-SHA:HIGH:MEDIUM:!aNULL:!MD5
    #SSLHonorCipherOrder on 

    #   Server Certificate:
    #   Point SSLCertificateFile at a PEM encoded certificate.  If
    #   the certificate is encrypted, then you will be prompted for a
    #   pass phrase.  Note that a kill -HUP will prompt again.  Keep
    #   in mind that if you have both an RSA and a DSA certificate you
    #   can configure both in parallel (to also allow the use of DSA
    #   ciphers, etc.)   --- default had 'server' where I have 'gremli
+n.site'
    SSLCertificateFile /etc/apache2/ssl.crt/gremlin.site.crt
    #SSLCertificateFile /etc/apache2/ssl.crt/server-dsa.crt

    #   Server Private Key:
    #   If the key is not combined with the certificate, use this
    #   directive to point at the key file.  Keep in mind that if
    #   you've both a RSA and a DSA private key you can configure
    #   both in parallel (to also allow the use of DSA ciphers, etc.)
    SSLCertificateKeyFile /etc/apache2/ssl.key/gremlin.site.key
    #SSLCertificateKeyFile /etc/apache2/ssl.key/server-dsa.key

    #   Server Certificate Chain:
    #   Point SSLCertificateChainFile at a file containing the
    #   concatenation of PEM encoded CA certificates which form the
    #   certificate chain for the server certificate. Alternatively
    #   the referenced file can be the same as SSLCertificateFile
    #   when the CA certificates are directly appended to the server
    #   certificate for convinience.
    #SSLCertificateChainFile /etc/apache2/ssl.crt/ca.crt

    #   Certificate Authority (CA):
    #   Set the CA certificate verification path where to find CA
    #   certificates for client authentication or alternatively one
    #   huge file containing all of them (file must be PEM encoded)
    #   Note: Inside SSLCACertificatePath you need hash symlinks
    #         to point to the certificate files. Use the provided
    #         Makefile to update the hash symlinks after changes.
    SSLCACertificatePath /etc/apache2/ssl.crt
    #SSLCACertificateFile /etc/apache2/ssl.crt/ca-bundle.crt
    SSLCACertificateFile /etc/apache2/ssl.crt/rootCA.pem

    #   Certificate Revocation Lists (CRL):
    #   Set the CA revocation path where to find CA CRLs for client
    #   authentication or alternatively one huge file containing all
    #   of them (file must be PEM encoded)
    #   Note: Inside SSLCARevocationPath you need hash symlinks
    #         to point to the certificate files. Use the provided
    #         Makefile to update the hash symlinks after changes.
    #SSLCARevocationPath /etc/apache2/ssl.crl
    #SSLCARevocationFile /etc/apache2/ssl.crl/ca-bundle.crl

    #   Client Authentication (Type):
    #   Client certificate verification type and depth.  Types are
    #   none, optional, require and optional_no_ca.  Depth is a
    #   number which specifies how deeply to verify the certificate
    #   issuer chain before deciding the certificate is not valid.
    #SSLVerifyClient require
    #SSLVerifyDepth  10

    #   Access Control:
    #   With SSLRequire you can do per-directory access control based
    #   on arbitrary complex boolean expressions containing server
    #   variable checks and other lookup directives.  The syntax is a
    #   mixture between C and Perl.  See the mod_ssl documentation
    #   for more details.
    #<Location />
    #SSLRequire (    %{SSL_CIPHER} !~ m/^(EXP|NULL)/ \
    #            and %{SSL_CLIENT_S_DN_O} eq "Snake Oil, Ltd." \
    #            and %{SSL_CLIENT_S_DN_OU} in {"Staff", "CA", "Dev"} \
    #            and %{TIME_WDAY} >= 1 and %{TIME_WDAY} <= 5 \
    #            and %{TIME_HOUR} >= 8 and %{TIME_HOUR} <= 20       ) 
+\
    #           or %{REMOTE_ADDR} =~ m/^192\.76\.162\.[0-9]+$/
    #</Location>

    #   SSL Engine Options:
    #   Set various options for the SSL engine.
    #   o FakeBasicAuth:
    #     Translate the client X.509 into a Basic Authorisation.  This
+ means that
    #     the standard Auth/DBMAuth methods can be used for access con
+trol.  The
    #     user name is the `one line' version of the client's X.509 ce
+rtificate.
    #     Note that no password is obtained from the user. Every entry
+ in the user
    #     file needs this password: `xxj31ZMTZzkVA'.
    #   o ExportCertData:
    #     This exports two additional environment variables: SSL_CLIEN
+T_CERT and
    #     SSL_SERVER_CERT. These contain the PEM-encoded certificates 
+of the
    #     server (always existing) and the client (only existing when 
+client
    #     authentication is used). This can be used to import the cert
+ificates
    #     into CGI scripts.
    #   o StdEnvVars:
    #     This exports the standard SSL/TLS related `SSL_*' environmen
+t variables.
    #     Per default this exportation is switched off for performance
+ reasons,
    #     because the extraction step is an expensive operation and is
+ usually
    #     useless for serving static content. So one usually enables t
+he
    #     exportation for CGI and SSI requests only.
    #   o StrictRequire:
    #     This denies access when "SSLRequireSSL" or "SSLRequire" appl
+ied even
    #     under a "Satisfy any" situation, i.e. when it applies access
+ is denied
    #     and no other module can change it.
    #   o OptRenegotiate:
    #     This enables optimized SSL connection renegotiation handling
+ when SSL
    #     directives are used in per-directory context. 
    #SSLOptions +FakeBasicAuth +ExportCertData +StrictRequire
    <FilesMatch "\.(cgi|shtml|phtml|php|pl)$">
        SSLOptions +StdEnvVars
    </FilesMatch>
    ScriptAlias /cgi-bin/ "/srv/www/vhosts/gremlin.site/cgi-bin/"
    <Directory "/srv/www/vhosts/gremlin.site/cgi-bin">
#    <Directory "/srv/www/cgi-bin">
           AllowOverride None
           Options +ExecCGI -Includes
           Order allow,deny
           Allow from all
               SSLOptions +ExportCertData +StdEnvVars
    </Directory>

    #   SSL Protocol Adjustments:
    #   The safe and default but still SSL/TLS standard compliant shut
+down
    #   approach is that mod_ssl sends the close notify alert but does
+n't wait for
    #   the close notify alert from client. When you need a different 
+shutdown
    #   approach you can use one of the following variables:
    #   o ssl-unclean-shutdown:
    #     This forces an unclean shutdown when the connection is close
+d, i.e. no
    #     SSL close notify alert is send or allowed to received.  This
+ violates
    #     the SSL/TLS standard but is needed for some brain-dead brows
+ers. Use
    #     this when you receive I/O errors because of the standard app
+roach where
    #     mod_ssl sends the close notify alert.
    #   o ssl-accurate-shutdown:
    #     This forces an accurate shutdown when the connection is clos
+ed, i.e. a
    #     SSL close notify alert is send and mod_ssl waits for the clo
+se notify
    #     alert of the client. This is 100% SSL/TLS standard compliant
+, but in
    #     practice often causes hanging connections with brain-dead br
+owsers. Use
    #     this only for browsers where you know that their SSL impleme
+ntation
    #     works correctly. 
    #   Notice: Most problems of broken clients are also related to th
+e HTTP
    #   keep-alive facility, so you usually additionally want to disab
+le
    #   keep-alive for those clients, too. Use variable "nokeepalive" 
+for this.
    #   Similarly, one has to force some clients to use HTTP/1.0 to wo
+rkaround
    #   their broken HTTP/1.1 implementation. Use variables "downgrade
+-1.0" and
    #   "force-response-1.0" for this.
    BrowserMatch "MSIE [2-5]" \
         nokeepalive ssl-unclean-shutdown \
         downgrade-1.0 force-response-1.0

    #   Per-Server Logging:
    #   The home of a custom SSL log file. Use this when you want a
    #   compact non-error SSL logfile on a virtual host basis.
    CustomLog /var/log/apache2/ssl_request_log   ssl_combined

</VirtualHost>                                  

</IfDefine>
</IfDefine>
[download]

And here is my package, intended to make handling this easier (eventually):

package REJBlibUA::client;

use strict;
use warnings;

use Net::SSL (); # From Crypt-SSLeay
use English qw(-no_match_vars);
use HTTP::Status;
use LWP::UserAgent;
use LWP::Protocol::https;
use HTTP::Request;
use HTTP::Request::Common;
use HTTP::Response;
use Log::Log4perl qw(:easy get_logger);
use UNIVERSAL::require;
use Encode qw(decode encode);

my $log_prefix = "[http client] ";

sub new {
    my ($class, %params) = @_;

    die "non-existing certificate file $params{ca_cert_file}"
        if $params{ca_cert_file} && ! -f $params{ca_cert_file};

    die "non-existing certificate directory $params{ca_cert_dir}"
        if $params{ca_cert_dir} && ! -d $params{ca_cert_dir};

    my $self = {
        logger       => '',
        user         => $params{user},
        password     => $params{password},
        timeout      => $params{timeout} || 180,
        ssl_set      => 0,
        no_ssl_check => $params{no_ssl_check},
        ca_cert_dir  => $params{ca_cert_dir},
        ca_cert_file => $params{ca_cert_file}
    };
    bless $self, $class;

    my $conf = q(
        log4perl.logger                    = TRACE, FileApp, ScreenApp
        log4perl.appender.FileApp          = Log::Log4perl::Appender::
+File
        log4perl.appender.FileApp.filename = lwp.log
        log4perl.appender.FileApp.layout   = PatternLayout
        log4perl.appender.FileApp.layout.ConversionPattern = %d> %m%n
        log4perl.appender.ScreenApp          = Log::Log4perl::Appender
+::Screen
        log4perl.appender.ScreenApp.stderr   = 0
        log4perl.appender.ScreenApp.layout   = PatternLayout
        log4perl.appender.ScreenApp.layout.ConversionPattern = %d> %m%
+n
    );

# Initialize logging behaviour
    Log::Log4perl->init( \$conf );
    Log::Log4perl->infiltrate_lwp();

    $self->{'logger'} = get_logger();
    # create user agent
    $self->{ua} = LWP::UserAgent->new(
        parse_head => 0, # No need to parse HTML
        keep_alive => 1,
        requests_redirectable => ['POST', 'GET', 'HEAD']
    );
    $self->{ua}->ssl_opts(verify_hostname => 0, SSL_verify_mode => 0);
    if ($params{proxy}) {
        $self->{ua}->proxy(['http', 'https'], $params{proxy});
    }  else {
        $self->{ua}->env_proxy();
    }

    $self->{ua}->timeout($self->{timeout});

    return $self;
}

sub request { 
   my ($self, $request, $file) = @_;
    # $request is a HTTP::Request object, created with only the URL
    # $file is a message, normally an XML file
    my $logger  = $self->{logger};

    my $url = $request->uri();
    my $scheme = $url->scheme();
   print "\$url = $url\n\t\$scheme = $scheme\n";
   print "\t\$self->{ssl_set} = ",$self->{ssl_set},"\n";
   print "\t\$self->{ca_cert_dir} = ",$self->{ca_cert_dir},"\n";
   print "\t\$self->{ca_cert_file} = ",$self->{ca_cert_file},"\n";
    $self->_setSSLOptions() if $scheme eq 'https' && !$self->{ssl_set}
+;

    my $result = HTTP::Response->new( 500 );
    eval {
       $result = $self->{ua}->request($request, $file);
    };
    # check result first
    if (!$result->is_success()) {
        # authentication required
        if ($result->code() == 401) {
            if ($self->{user} && $self->{password}) {
                $logger->debug(
                    $log_prefix .
                    "authentication required, submitting credentials"
                );
                # compute authentication parameters
                my $header = $result->header('www-authenticate');
                my ($realm) = $header =~ /^Basic realm="(.*)"/;
                my $host = $url->host();
                my $port = $url->port() ||
                   ($scheme eq 'https' ? 443 : 80);
                $self->{ua}->credentials(
                    "$host:$port",
                    $realm,
                    $self->{user},
                    $self->{password}
                );
                # replay request
                eval {
                    if ($OSNAME eq 'MSWin32' && $scheme eq 'https') {
                        alarm $self->{timeout};
                    }
                    $result = $self->{ua}->request($request, $file);
                };
                if (!$result->is_success()) {
                    $logger->error(
                        $log_prefix .
                        "authentication required, wrong credentials"
                    );
                }
            } else {
                # abort
                $logger->error(
                    $log_prefix .
                    "authentication required, no credentials available
+"
                );
            }
        } else {
            $logger->error(
                $log_prefix .
                "communication error: " . $result->status_line()
            );
        }
    }

    return $result;
}

sub _setSSLOptions {
    my ($self) = @_;

    # SSL handling
    if ($self->{no_ssl_check}) {
       # LWP 6 default behaviour is to check hostname
       # Fedora also backported this behaviour change in its LWP5 pack
+age, so
       # just checking on LWP version is not enough
       $self->{ua}->ssl_opts(verify_hostname => 0, SSL_verify_mode => 
+0)
           if $self->{ua}->can('ssl_opts');
    } else {
        # only IO::Socket::SSL can perform full server certificate val
+idation,
        # Net::SSL is only able to check certification authority, and 
+not
        # certificate hostname
        IO::Socket::SSL->require();
        die
            "failed to load IO::Socket::SSL, "                 .
            "unable to perform SSL certificate validation.\n"  .
            "You can use 'no-ssl-check' option to disable it."
            if $EVAL_ERROR;

#        if ($self->{logger}{debug} >= 3) {
#            $Net::SSLeay::trace = 2;
#        }
    print "\t\t\$LWP::VERSION = $LWP::VERSION\n";

        if ($LWP::VERSION >= 6) {
        print "\t\tSetting cert dir and file if available\n";
            $self->{ua}->ssl_opts(SSL_ca_file => $self->{ca_cert_file}
+)
                if $self->{ca_cert_file};
            $self->{ua}->ssl_opts(SSL_ca_path => $self->{ca_cert_dir})
                if $self->{ca_cert_dir};
        }
    }

    $self->{ssl_set} = 1;
}

1;
[download]

And here is the test script:

#!/usr/bin/perl

use HTTP::Request;
use HTTP::Response;

use lib './Work';

use REJBlibUA::client;

my $method = "POST";
my @requests = (HTTP::Request->new( $method,'https://www.google.ca'), 
                HTTP::Request->new( $method,'https://gremlin.site/cgi-
+bin/printenv.pl'),
        HTTP::Request->new( $method,'https://byerspublishing.com'));

my $cnt = 0;
foreach my $r (@requests) {
    my %rp;
    if ($cnt == 1) {
    $rp{'ca_cert_file'} = 'client.crt';
    $rp{'ca_cert_dir'} = '.';
    }

    my $c = REJBlibUA::client->new(%rp);
    
    my $resp = $c->request($r);

    if ($resp->is_success) {
    print $resp->decoded_content;
    } else {
    print STDERR $resp->status_line, "\n";
    }
    $cnt += 1;
}
[download]

NB: both client and the gremlin.site server are OpenSuse 13.1 boxes, and byerspublishing.com (a private server, visible only within my LAN for now) is on Ubuntu 12.04 and is not configured to want client side certificates,

Note line 64 in my client package: $self->{ua}->ssl_opts(verify_hostname => 0, SSL_verify_mode => 0); The values of 0 and 0 are the only values that lets my test script get data. If I set only SSL_verify_mode to 1, I get the following error:

2014/07/31 11:42:21> [http client] communication error: 500 SSL negoti
+ation failed: 
500 SSL negotiation failed:
[download]

If I set only verify_hostname to 1, I get the following error:

2014/07/31 11:44:14> [http client] communication error: 500 Can't conn
+ect to gremlin.site:443 (Crypt-SSLeay can't verify hostnames)
500 Can't connect to gremlin.site:443 (Crypt-SSLeay can't verify hostn
+ames)
[download]

Athough above I show only one error in each case, I get the same error in each case for ALL the hosts I try to reach. NB: Setting both of these parameters to 1 gets the same error that verify_hostname produces.

Now, curiously, if I comment out "use Net::SSL (); # From Crypt-SSLeay", the error changes, when both parameters are set to 1.

2014/07/31 11:48:24> [http client] communication error: 500 Can't conn
+ect to gremlin.site:443 (certificate verify failed)
500 Can't connect to gremlin.site:443 (certificate verify failed)
[download]

I can understand this for byerspublishing.com as it is using only the test certificate provided when one installs Apache2. But this error makes no sense gor gremlin.site, as the ONLY certificates in the relevant directories for Apache are those I created myelf, as I created a rootCA using openssl, and then I used that rootCA to make the certificates both for the server and for the client. So why, in the case of gremlin.site, would there be a problem verifying the certificates? Is the cetificate validation error generated by the server or the client?

I'd appreciate any help you can provide in solving this problem. Any tips for improving client.pm would be appreciated.

Thanks

Ted

Comment on How do I ensure client side certificates are used in SSL handshaking? Select or Download Code

Replies are listed 'Best First'.
Re: How do I ensure client side certificates are used in SSL handshaking? by ted.byers (Monk) on Jul 31, 2014 at 20:03 UTC
Problem solved The confusion I experienced was due mostly to conflicting and often misleading content on web pages turned up by google; and in part to the lack of a comprehensive guide to using LWP, particularly with regard to HTTPS, in such a manner that SSL/TLS hand shaking is done correctly both with and without client side certificates. Perhaps someone who understands encrypted data transfer between client and server much better than I can put something together. To be clear, LWP's documentation, and that for related packages, is generally good, in order for one to start, but it is too terse, and provides too little discussion of the complexities of handling SSL/TLS properly, and in the most secure manner possible. That said, I finally found a page that actually shows how to have useragent send client side certificates to the server. To provide useful information to others who may be struggling with this, here is my client package: package REJBlibUA::client; use strict; use warnings; #use Net::SSL (); use English qw(-no_match_vars); use HTTP::Status; use LWP::UserAgent; use LWP::Protocol::https; use HTTP::Request; use HTTP::Request::Common; use HTTP::Response; use Log::Log4perl qw(:easy get_logger); use UNIVERSAL::require; use Encode qw(decode encode); my $log_prefix = "[http client] "; sub new { my ($class, %params) = @_; die "non-existing certificate file $params{ca_cert_file}" if $params{ca_cert_file} && ! -f $params{ca_cert_file}; die "non-existing certificate directory $params{ca_cert_dir}" if $params{ca_cert_dir} && ! -d $params{ca_cert_dir}; my $self = { logger => '', user => $params{user}, password => $params{password}, timeout => $params{timeout} \|\| 180, ssl_set => 0, no_ssl_check => $params{no_ssl_check}, ca_cert_dir => $params{ca_cert_dir}, ca_cert_file => $params{ca_cert_file}, SSL_cert_file => $params{SSL_cert_file}, SSL_key_file => $params{SSL_key_file}, }; bless $self, $class; my $conf = q( log4perl.logger = TRACE, FileApp, ScreenApp log4perl.appender.FileApp = Log::Log4perl::Appender:: +File log4perl.appender.FileApp.filename = lwp.log log4perl.appender.FileApp.layout = PatternLayout log4perl.appender.FileApp.layout.ConversionPattern = %d> %m%n log4perl.appender.ScreenApp = Log::Log4perl::Appender +::Screen log4perl.appender.ScreenApp.stderr = 0 log4perl.appender.ScreenApp.layout = PatternLayout log4perl.appender.ScreenApp.layout.ConversionPattern = %d> %m% +n ); # Initialize logging behaviour Log::Log4perl->init( \$conf ); Log::Log4perl->infiltrate_lwp(); $self->{'logger'} = get_logger(); # create user agent $self->{ua} = LWP::UserAgent->new( parse_head => 0, # No need to parse HTML keep_alive => 1, requests_redirectable => ['POST', 'GET', 'HEAD'] ); $self->{ua}->ssl_opts(verify_hostname => 1, SSL_verify_mode => 1); if ($params{proxy}) { $self->{ua}->proxy(['http', 'https'], $params{proxy}); } else { $self->{ua}->env_proxy(); } $self->{ua}->timeout($self->{timeout}); return $self; } sub request { my ($self, $request, $file) = @_; # $request is a HTTP::Request object, created with only the URL # $file is a message, normally an XML file my $logger = $self->{logger}; my $url = $request->uri(); my $scheme = $url->scheme(); print "\$url = $url\n\t\$scheme = $scheme\n"; print "\t\$self->{ssl_set} = ",$self->{ssl_set},"\n"; print "\t\$self->{ca_cert_dir} = ",$self->{ca_cert_dir},"\n"; print "\t\$self->{ca_cert_file} = ",$self->{ca_cert_file},"\n"; $self->_setSSLOptions() if $scheme eq 'https' && !$self->{ssl_set} +; my $result = HTTP::Response->new( 500 ); eval { $result = $self->{ua}->request($request, $file); }; # check result first if (!$result->is_success()) { # authentication required if ($result->code() == 401) { if ($self->{user} && $self->{password}) { $logger->debug( $log_prefix . "authentication required, submitting credentials" ); # compute authentication parameters my $header = $result->header('www-authenticate'); my ($realm) = $header =~ /^Basic realm="(.*)"/; my $host = $url->host(); my $port = $url->port() \|\| ($scheme eq 'https' ? 443 : 80); $self->{ua}->credentials( "$host:$port", $realm, $self->{user}, $self->{password} ); # replay request eval { if ($OSNAME eq 'MSWin32' && $scheme eq 'https') { alarm $self->{timeout}; } $result = $self->{ua}->request($request, $file); }; if (!$result->is_success()) { $logger->error( $log_prefix . "authentication required, wrong credentials" ); } } else { # abort $logger->error( $log_prefix . "authentication required, no credentials available +" ); } } else { $logger->error( $log_prefix . "communication error: " . $result->status_line() ); } } return $result; } sub _setSSLOptions { my ($self) = @_; # SSL handling if ($self->{no_ssl_check}) { # LWP 6 default behaviour is to check hostname # Fedora also backported this behaviour change in its LWP5 pack +age, so # just checking on LWP version is not enough $self->{ua}->ssl_opts(verify_hostname => 0, SSL_verify_mode => +0) if $self->{ua}->can('ssl_opts'); } else { # only IO::Socket::SSL can perform full server certificate val +idation, # Net::SSL is only able to check certification authority, and +not # certificate hostname IO::Socket::SSL->require(); die "failed to load IO::Socket::SSL, " . "unable to perform SSL certificate validation.\n" . "You can use 'no-ssl-check' option to disable it." if $EVAL_ERROR; # if ($self->{logger}{debug} >= 3) { # $Net::SSLeay::trace = 2; # } print "\t\t\$LWP::VERSION = $LWP::VERSION\n"; if ($LWP::VERSION >= 6) { print "\t\tSetting cert dir and file if available\n"; $self->{ua}->ssl_opts(SSL_ca_file => $self->{ca_cert_file} +) if $self->{ca_cert_file}; $self->{ua}->ssl_opts(SSL_ca_path => $self->{ca_cert_dir}) if $self->{ca_cert_dir}; $self->{ua}->ssl_opts(SSL_cert_file => $self->{SSL_cert_fi +le}) if $self->{SSL_cert_file}; $self->{ua}->ssl_opts(SSL_key_file => $self->{SSL_key_file +}) if $self->{SSL_key_file}; } } $self->{ssl_set} = 1; } 1; [download] And here is a script that, partially, tests it: #!/usr/bin/perl use HTTP::Request; use HTTP::Response; use lib './Work'; use REJBlibUA::client; my $method = "POST"; my @requests = (HTTP::Request->new( $method,'https://www.google.ca'), HTTP::Request->new( $method,'https://gremlin.site/cgi- +bin/printenv.pl'), HTTP::Request->new( $method,'https://byerspublishing.com')); my $cnt = 0; foreach my $r (@requests) { my %rp; if ($cnt == 1) { $rp{'ca_cert_file'} = 'rootCA.pem'; # $rp{'ca_cert_dir'} = '.'; $rp{'SSL_cert_file'} = 'client.crt'; $rp{'SSL_key_file'} = 'client.key'; } my $c = REJBlibUA::client->new(%rp); my $resp = $c->request($r); if ($resp->is_success) { print $resp->decoded_content; } else { print STDERR $resp->status_line, "\n"; } $cnt += 1; } [download] The code isn't perfect, as it has yet to get anything from google.ca. Instead, it gives this: `$url = https://www.google.ca $scheme = https $self->{ssl_set} = 0 Use of uninitialized value in print at Work/REJBlibUA/client.pm line 8 +8. $self->{ca_cert_dir} = Use of uninitialized value in print at Work/REJBlibUA/client.pm line 8 +9. $self->{ca_cert_file} = $LWP::VERSION = 6.05 Setting cert dir and file if available 2014/07/31 15:15:46> [http client] communication error: 411 Length Req +uired` [download] While I do not know what that means, or how to fix it, I am not too too worried about it, at present, as the important thing for the present, for me, is getting SSL/TLS handled properly. It would be helpful if you could review my code and advise as to whether it is correct/complete, or whether there are improvements that can be made. For example, what I'd like to do is move the code in the test script relating to HTTP::Request and HTTP::Response into client.pm, and have request return a string or a hash or an XML file, based on what the server returns, and have it take, as an argument, the url and a hash of request parameters, or an XML file containing the details of what is requested of the server (like SOAP). An ideal objective is to move everything related to the details of communicating with a server from the calling code. I suppose that means constructing a more robust means of collecting errors that can occur and return appropriate information about filures to the calling code. Ideas are welcome. ;-) Thanks Ted	[reply] [d/l] [select]
Re^2: How do I ensure client side certificates are used in SSL handshaking? by ww (Archbishop) on Jul 31, 2014 at 23:15 UTC
It's truly wonderous how explaining your problems to your teddy bear will help resolve those problems. check Ln42!	[reply]
Re: How do I ensure client side certificates are used in SSL handshaking? by ted.byers (Monk) on Jul 31, 2014 at 18:36 UTC
Here is something that bears on the problem at hand, but I do not know how to act on it. Check out this link: http://search.cpan.org/~sullr/IO-Socket-SSL-1.997/lib/IO/Socket/SSL.pod. There I read the following: "Check if we trust the certificate, e.g. make sure its not a forgery. "We believe that a certificate is not a fake, if we either know the certificate already or if we trust the issuer (the CA) and can verify the issuers signature on the certificate. In reality there is often a hierarchy of certificate agencies and we only directly trust the root of this hierarchy. In this case the peer not only sends his own certificate, but also all intermediate certificates. Verification will be done by building a trust path from the trusted root up to the peers certificate and checking in each step if the we can verify the issuers signature. "This step often causes problems, because the client does not know the necessary trusted root certificates. These are usually stored in a system dependent CA store, but often the browsers have their own CA store." Emphasis added. Both the server and client side certificates were signed by the same CA (one I created using openssl), and I have the CA's certificate on both machines. The CA's certificate is in the proper directory for apache to use it if it wishes, but how do I tell my client software about it. I have not (yet) found a way to tell LWP's user agent about the CA's (my) certificate, Might this problem be a client side problem. If so, how do I tell the user agent on my client machine to use my CA's certificate for validating my server's certificate? Also, do I have to worry about telling servers like nginx or apache to send the whole both the server's certificate and the whole change of intermediate certificates, or do they do that automagically? But that begs the question of how to do that in my client side code? Is that by placing the CA root's certificate on the server and configuring the server to use it when checking the client side certificates? Or do I have to add more code to tell the user agent to send them? Thanks Ted	[reply]
Re^2: How do I ensure client side certificates are used in SSL handshaking? by ted.byers (Monk) on Jul 31, 2014 at 18:57 UTC
This lead me to a fundamental misunderstanding of the ssl options was using. The options I was using are for the CA, not the client side certificate. Thus, this changes to, "How do I provide the client side certificate?"</P Sorry for my confusion. Ted	[reply]
Re: How do I ensure client side certificates are used in SSL handshaking? by ted.byers (Monk) on Jul 31, 2014 at 18:07 UTC
One thing I didn't mention, and should have, is this: I have often seen, as a result of googling error messages, requests for direction on how to ignore some of these error messages. I am not interesting in ignoring error messages, but rather, in ensuring that both server and client side certificates are properly handled, and that the SSL/TLS handshaking to establish the connection is handled correctly, and in the most secure manner possible. It seems to me that ignoring these errors probably creates the illusion of secure communications while creating real security vulnerabilities. I am no expert in SSL/TLS, so correct me if I am wrong, but that is the view I willl act on until I see evidence to the contrary. Thanks Ted	[reply]