Problem solved
The confusion I experienced was due mostly to conflicting and often misleading content on web pages turned up by google; and in part to the lack of a comprehensive guide to using LWP, particularly with regard to HTTPS, in such a manner that SSL/TLS hand shaking is done correctly both with and without client side certificates. Perhaps someone who understands encrypted data transfer between client and server much better than I can put something together. To be clear, LWP's documentation, and that for related packages, is generally good, in order for one to start, but it is too terse, and provides too little discussion of the complexities of handling SSL/TLS properly, and in the most secure manner possible.
That said, I finally found a page that actually shows how to have useragent send client side certificates to the server. To provide useful information to others who may be struggling with this, here is my client package:
package REJBlibUA::client; use strict; use warnings; #use Net::SSL (); use English qw(-no_match_vars); use HTTP::Status; use LWP::UserAgent; use LWP::Protocol::https; use HTTP::Request; use HTTP::Request::Common; use HTTP::Response; use Log::Log4perl qw(:easy get_logger); use UNIVERSAL::require; use Encode qw(decode encode); my $log_prefix = "[http client] "; sub new { my ($class, %params) = @_; die "non-existing certificate file $params{ca_cert_file}" if $params{ca_cert_file} && ! -f $params{ca_cert_file}; die "non-existing certificate directory $params{ca_cert_dir}" if $params{ca_cert_dir} && ! -d $params{ca_cert_dir}; my $self = { logger => '', user => $params{user}, password => $params{password}, timeout => $params{timeout} || 180, ssl_set => 0, no_ssl_check => $params{no_ssl_check}, ca_cert_dir => $params{ca_cert_dir}, ca_cert_file => $params{ca_cert_file}, SSL_cert_file => $params{SSL_cert_file}, SSL_key_file => $params{SSL_key_file}, }; bless $self, $class; my $conf = q( log4perl.logger = TRACE, FileApp, ScreenApp log4perl.appender.FileApp = Log::Log4perl::Appender:: +File log4perl.appender.FileApp.filename = lwp.log log4perl.appender.FileApp.layout = PatternLayout log4perl.appender.FileApp.layout.ConversionPattern = %d> %m%n log4perl.appender.ScreenApp = Log::Log4perl::Appender +::Screen log4perl.appender.ScreenApp.stderr = 0 log4perl.appender.ScreenApp.layout = PatternLayout log4perl.appender.ScreenApp.layout.ConversionPattern = %d> %m% +n ); # Initialize logging behaviour Log::Log4perl->init( \$conf ); Log::Log4perl->infiltrate_lwp(); $self->{'logger'} = get_logger(); # create user agent $self->{ua} = LWP::UserAgent->new( parse_head => 0, # No need to parse HTML keep_alive => 1, requests_redirectable => ['POST', 'GET', 'HEAD'] ); $self->{ua}->ssl_opts(verify_hostname => 1, SSL_verify_mode => 1); if ($params{proxy}) { $self->{ua}->proxy(['http', 'https'], $params{proxy}); } else { $self->{ua}->env_proxy(); } $self->{ua}->timeout($self->{timeout}); return $self; } sub request { my ($self, $request, $file) = @_; # $request is a HTTP::Request object, created with only the URL # $file is a message, normally an XML file my $logger = $self->{logger}; my $url = $request->uri(); my $scheme = $url->scheme(); print "\$url = $url\n\t\$scheme = $scheme\n"; print "\t\$self->{ssl_set} = ",$self->{ssl_set},"\n"; print "\t\$self->{ca_cert_dir} = ",$self->{ca_cert_dir},"\n"; print "\t\$self->{ca_cert_file} = ",$self->{ca_cert_file},"\n"; $self->_setSSLOptions() if $scheme eq 'https' && !$self->{ssl_set} +; my $result = HTTP::Response->new( 500 ); eval { $result = $self->{ua}->request($request, $file); }; # check result first if (!$result->is_success()) { # authentication required if ($result->code() == 401) { if ($self->{user} && $self->{password}) { $logger->debug( $log_prefix . "authentication required, submitting credentials" ); # compute authentication parameters my $header = $result->header('www-authenticate'); my ($realm) = $header =~ /^Basic realm="(.*)"/; my $host = $url->host(); my $port = $url->port() || ($scheme eq 'https' ? 443 : 80); $self->{ua}->credentials( "$host:$port", $realm, $self->{user}, $self->{password} ); # replay request eval { if ($OSNAME eq 'MSWin32' && $scheme eq 'https') { alarm $self->{timeout}; } $result = $self->{ua}->request($request, $file); }; if (!$result->is_success()) { $logger->error( $log_prefix . "authentication required, wrong credentials" ); } } else { # abort $logger->error( $log_prefix . "authentication required, no credentials available +" ); } } else { $logger->error( $log_prefix . "communication error: " . $result->status_line() ); } } return $result; } sub _setSSLOptions { my ($self) = @_; # SSL handling if ($self->{no_ssl_check}) { # LWP 6 default behaviour is to check hostname # Fedora also backported this behaviour change in its LWP5 pack +age, so # just checking on LWP version is not enough $self->{ua}->ssl_opts(verify_hostname => 0, SSL_verify_mode => +0) if $self->{ua}->can('ssl_opts'); } else { # only IO::Socket::SSL can perform full server certificate val +idation, # Net::SSL is only able to check certification authority, and +not # certificate hostname IO::Socket::SSL->require(); die "failed to load IO::Socket::SSL, " . "unable to perform SSL certificate validation.\n" . "You can use 'no-ssl-check' option to disable it." if $EVAL_ERROR; # if ($self->{logger}{debug} >= 3) { # $Net::SSLeay::trace = 2; # } print "\t\t\$LWP::VERSION = $LWP::VERSION\n"; if ($LWP::VERSION >= 6) { print "\t\tSetting cert dir and file if available\n"; $self->{ua}->ssl_opts(SSL_ca_file => $self->{ca_cert_file} +) if $self->{ca_cert_file}; $self->{ua}->ssl_opts(SSL_ca_path => $self->{ca_cert_dir}) if $self->{ca_cert_dir}; $self->{ua}->ssl_opts(SSL_cert_file => $self->{SSL_cert_fi +le}) if $self->{SSL_cert_file}; $self->{ua}->ssl_opts(SSL_key_file => $self->{SSL_key_file +}) if $self->{SSL_key_file}; } } $self->{ssl_set} = 1; } 1;
And here is a script that, partially, tests it:
#!/usr/bin/perl use HTTP::Request; use HTTP::Response; use lib './Work'; use REJBlibUA::client; my $method = "POST"; my @requests = (HTTP::Request->new( $method,'https://www.google.ca'), HTTP::Request->new( $method,'https://gremlin.site/cgi- +bin/printenv.pl'), HTTP::Request->new( $method,'https://byerspublishing.com')); my $cnt = 0; foreach my $r (@requests) { my %rp; if ($cnt == 1) { $rp{'ca_cert_file'} = 'rootCA.pem'; # $rp{'ca_cert_dir'} = '.'; $rp{'SSL_cert_file'} = 'client.crt'; $rp{'SSL_key_file'} = 'client.key'; } my $c = REJBlibUA::client->new(%rp); my $resp = $c->request($r); if ($resp->is_success) { print $resp->decoded_content; } else { print STDERR $resp->status_line, "\n"; } $cnt += 1; }
The code isn't perfect, as it has yet to get anything from google.ca. Instead, it gives this:
$url = https://www.google.ca $scheme = https $self->{ssl_set} = 0 Use of uninitialized value in print at Work/REJBlibUA/client.pm line 8 +8. $self->{ca_cert_dir} = Use of uninitialized value in print at Work/REJBlibUA/client.pm line 8 +9. $self->{ca_cert_file} = $LWP::VERSION = 6.05 Setting cert dir and file if available 2014/07/31 15:15:46> [http client] communication error: 411 Length Req +uired
While I do not know what that means, or how to fix it, I am not too too worried about it, at present, as the important thing for the present, for me, is getting SSL/TLS handled properly. It would be helpful if you could review my code and advise as to whether it is correct/complete, or whether there are improvements that can be made. For example, what I'd like to do is move the code in the test script relating to HTTP::Request and HTTP::Response into client.pm, and have request return a string or a hash or an XML file, based on what the server returns, and have it take, as an argument, the url and a hash of request parameters, or an XML file containing the details of what is requested of the server (like SOAP). An ideal objective is to move everything related to the details of communicating with a server from the calling code. I suppose that means constructing a more robust means of collecting errors that can occur and return appropriate information about filures to the calling code. Ideas are welcome. ;-)
Thanks
Ted
In reply to Re: How do I ensure client side certificates are used in SSL handshaking?
by ted.byers
in thread How do I ensure client side certificates are used in SSL handshaking?
by ted.byers
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |