I have SSL working, mostly, when communicating with a server on my own LAN (not publicly accessable). My browsers can connect to it using HTTPS. But, as you're aware, the usual printenv.pl script will dispay the contents of any certificates, both client and server and when I use LWP's user agent, and provide the client certificate file name, and the directory it is in, the printenv.pl script produces output indicating that the client side certificate is empty.
Here is how the SSL virtual host is configured:
# Template for a VirtualHost with SSL # Note: to use the template, rename it to /etc/apache2/vhost.d/yourvho +st.conf. # Files must have the .conf suffix to be loaded. # # See /usr/share/doc/packages/apache2/README.QUICKSTART for further hi +nts # about virtual hosts. # NameVirtualHost statements should be added to /etc/apache2/listen.co +nf. # # This is the Apache server configuration file providing SSL support. # It contains the configuration directives to instruct the server how +to # serve pages over an https connection. For detailing information abou +t these # directives see http://httpd.apache.org/docs/2.2/mod/mod_ssl.html # # Do NOT simply read the instructions in here without understanding # what they do. They're here only as hints or reminders. If you are +unsure # consult the online docs. You have been warned. # <IfDefine SSL> <IfDefine !NOSSL> ## ## SSL Virtual Host Context ## <VirtualHost _default_:443 > ServerAdmin r.ted.byers@gmail.com ServerName gremlin.site DocumentRoot /srv/www/vhosts/gremlin.site # General setup for the virtual host #DocumentRoot "/srv/www/htdocs" #ServerName www.example.com:443 #ServerAdmin webmaster@example.com # ErrorLog /var/log/apache2/error_log # TransferLog /var/log/apache2/access_log ErrorLog /var/log/apache2/gremlin.site-error_log CustomLog /var/log/apache2/gremlin.site-access_log combined # SSL Engine Switch: # Enable/Disable SSL for this virtual host. SSLEngine on # SSL protocols # Supporting TLS only is adequate nowadays SSLProtocol all -SSLv2 # SSL Cipher Suite: # List the ciphers that the client is permitted to negotiate. # See the mod_ssl documentation for a complete list. SSLCipherSuite HIGH:MEDIUM:!aNULL:!MD5 SSLVerifyClient optional SSLVerifyDepth 10 SSLOptions +ExportCertData +StdEnvVars # Speed-optimized SSL Cipher configuration: # If speed is your main concern (on busy HTTPS servers e.g.), # you might want to force clients to specific, performance # optimized ciphers. In this case, prepend those ciphers # to the SSLCipherSuite list, and enable SSLHonorCipherOrder. # Caveat: by giving precedence to RC4-SHA and AES128-SHA # (as in the example below), most connections will no longer # have perfect forward secrecy - if the server's key is # compromised, captures of past or future traffic must be # considered compromised, too. #SSLCipherSuite RC4-SHA:AES128-SHA:HIGH:MEDIUM:!aNULL:!MD5 #SSLHonorCipherOrder on # Server Certificate: # Point SSLCertificateFile at a PEM encoded certificate. If # the certificate is encrypted, then you will be prompted for a # pass phrase. Note that a kill -HUP will prompt again. Keep # in mind that if you have both an RSA and a DSA certificate you # can configure both in parallel (to also allow the use of DSA # ciphers, etc.) --- default had 'server' where I have 'gremli +n.site' SSLCertificateFile /etc/apache2/ssl.crt/gremlin.site.crt #SSLCertificateFile /etc/apache2/ssl.crt/server-dsa.crt # Server Private Key: # If the key is not combined with the certificate, use this # directive to point at the key file. Keep in mind that if # you've both a RSA and a DSA private key you can configure # both in parallel (to also allow the use of DSA ciphers, etc.) SSLCertificateKeyFile /etc/apache2/ssl.key/gremlin.site.key #SSLCertificateKeyFile /etc/apache2/ssl.key/server-dsa.key # Server Certificate Chain: # Point SSLCertificateChainFile at a file containing the # concatenation of PEM encoded CA certificates which form the # certificate chain for the server certificate. Alternatively # the referenced file can be the same as SSLCertificateFile # when the CA certificates are directly appended to the server # certificate for convinience. #SSLCertificateChainFile /etc/apache2/ssl.crt/ca.crt # Certificate Authority (CA): # Set the CA certificate verification path where to find CA # certificates for client authentication or alternatively one # huge file containing all of them (file must be PEM encoded) # Note: Inside SSLCACertificatePath you need hash symlinks # to point to the certificate files. Use the provided # Makefile to update the hash symlinks after changes. SSLCACertificatePath /etc/apache2/ssl.crt #SSLCACertificateFile /etc/apache2/ssl.crt/ca-bundle.crt SSLCACertificateFile /etc/apache2/ssl.crt/rootCA.pem # Certificate Revocation Lists (CRL): # Set the CA revocation path where to find CA CRLs for client # authentication or alternatively one huge file containing all # of them (file must be PEM encoded) # Note: Inside SSLCARevocationPath you need hash symlinks # to point to the certificate files. Use the provided # Makefile to update the hash symlinks after changes. #SSLCARevocationPath /etc/apache2/ssl.crl #SSLCARevocationFile /etc/apache2/ssl.crl/ca-bundle.crl # Client Authentication (Type): # Client certificate verification type and depth. Types are # none, optional, require and optional_no_ca. Depth is a # number which specifies how deeply to verify the certificate # issuer chain before deciding the certificate is not valid. #SSLVerifyClient require #SSLVerifyDepth 10 # Access Control: # With SSLRequire you can do per-directory access control based # on arbitrary complex boolean expressions containing server # variable checks and other lookup directives. The syntax is a # mixture between C and Perl. See the mod_ssl documentation # for more details. #<Location /> #SSLRequire ( %{SSL_CIPHER} !~ m/^(EXP|NULL)/ \ # and %{SSL_CLIENT_S_DN_O} eq "Snake Oil, Ltd." \ # and %{SSL_CLIENT_S_DN_OU} in {"Staff", "CA", "Dev"} \ # and %{TIME_WDAY} >= 1 and %{TIME_WDAY} <= 5 \ # and %{TIME_HOUR} >= 8 and %{TIME_HOUR} <= 20 ) +\ # or %{REMOTE_ADDR} =~ m/^192\.76\.162\.[0-9]+$/ #</Location> # SSL Engine Options: # Set various options for the SSL engine. # o FakeBasicAuth: # Translate the client X.509 into a Basic Authorisation. This + means that # the standard Auth/DBMAuth methods can be used for access con +trol. The # user name is the `one line' version of the client's X.509 ce +rtificate. # Note that no password is obtained from the user. Every entry + in the user # file needs this password: `xxj31ZMTZzkVA'. # o ExportCertData: # This exports two additional environment variables: SSL_CLIEN +T_CERT and # SSL_SERVER_CERT. These contain the PEM-encoded certificates +of the # server (always existing) and the client (only existing when +client # authentication is used). This can be used to import the cert +ificates # into CGI scripts. # o StdEnvVars: # This exports the standard SSL/TLS related `SSL_*' environmen +t variables. # Per default this exportation is switched off for performance + reasons, # because the extraction step is an expensive operation and is + usually # useless for serving static content. So one usually enables t +he # exportation for CGI and SSI requests only. # o StrictRequire: # This denies access when "SSLRequireSSL" or "SSLRequire" appl +ied even # under a "Satisfy any" situation, i.e. when it applies access + is denied # and no other module can change it. # o OptRenegotiate: # This enables optimized SSL connection renegotiation handling + when SSL # directives are used in per-directory context. #SSLOptions +FakeBasicAuth +ExportCertData +StrictRequire <FilesMatch "\.(cgi|shtml|phtml|php|pl)$"> SSLOptions +StdEnvVars </FilesMatch> ScriptAlias /cgi-bin/ "/srv/www/vhosts/gremlin.site/cgi-bin/" <Directory "/srv/www/vhosts/gremlin.site/cgi-bin"> # <Directory "/srv/www/cgi-bin"> AllowOverride None Options +ExecCGI -Includes Order allow,deny Allow from all SSLOptions +ExportCertData +StdEnvVars </Directory> # SSL Protocol Adjustments: # The safe and default but still SSL/TLS standard compliant shut +down # approach is that mod_ssl sends the close notify alert but does +n't wait for # the close notify alert from client. When you need a different +shutdown # approach you can use one of the following variables: # o ssl-unclean-shutdown: # This forces an unclean shutdown when the connection is close +d, i.e. no # SSL close notify alert is send or allowed to received. This + violates # the SSL/TLS standard but is needed for some brain-dead brows +ers. Use # this when you receive I/O errors because of the standard app +roach where # mod_ssl sends the close notify alert. # o ssl-accurate-shutdown: # This forces an accurate shutdown when the connection is clos +ed, i.e. a # SSL close notify alert is send and mod_ssl waits for the clo +se notify # alert of the client. This is 100% SSL/TLS standard compliant +, but in # practice often causes hanging connections with brain-dead br +owsers. Use # this only for browsers where you know that their SSL impleme +ntation # works correctly. # Notice: Most problems of broken clients are also related to th +e HTTP # keep-alive facility, so you usually additionally want to disab +le # keep-alive for those clients, too. Use variable "nokeepalive" +for this. # Similarly, one has to force some clients to use HTTP/1.0 to wo +rkaround # their broken HTTP/1.1 implementation. Use variables "downgrade +-1.0" and # "force-response-1.0" for this. BrowserMatch "MSIE [2-5]" \ nokeepalive ssl-unclean-shutdown \ downgrade-1.0 force-response-1.0 # Per-Server Logging: # The home of a custom SSL log file. Use this when you want a # compact non-error SSL logfile on a virtual host basis. CustomLog /var/log/apache2/ssl_request_log ssl_combined </VirtualHost> </IfDefine> </IfDefine>
And here is my package, intended to make handling this easier (eventually):
package REJBlibUA::client; use strict; use warnings; use Net::SSL (); # From Crypt-SSLeay use English qw(-no_match_vars); use HTTP::Status; use LWP::UserAgent; use LWP::Protocol::https; use HTTP::Request; use HTTP::Request::Common; use HTTP::Response; use Log::Log4perl qw(:easy get_logger); use UNIVERSAL::require; use Encode qw(decode encode); my $log_prefix = "[http client] "; sub new { my ($class, %params) = @_; die "non-existing certificate file $params{ca_cert_file}" if $params{ca_cert_file} && ! -f $params{ca_cert_file}; die "non-existing certificate directory $params{ca_cert_dir}" if $params{ca_cert_dir} && ! -d $params{ca_cert_dir}; my $self = { logger => '', user => $params{user}, password => $params{password}, timeout => $params{timeout} || 180, ssl_set => 0, no_ssl_check => $params{no_ssl_check}, ca_cert_dir => $params{ca_cert_dir}, ca_cert_file => $params{ca_cert_file} }; bless $self, $class; my $conf = q( log4perl.logger = TRACE, FileApp, ScreenApp log4perl.appender.FileApp = Log::Log4perl::Appender:: +File log4perl.appender.FileApp.filename = lwp.log log4perl.appender.FileApp.layout = PatternLayout log4perl.appender.FileApp.layout.ConversionPattern = %d> %m%n log4perl.appender.ScreenApp = Log::Log4perl::Appender +::Screen log4perl.appender.ScreenApp.stderr = 0 log4perl.appender.ScreenApp.layout = PatternLayout log4perl.appender.ScreenApp.layout.ConversionPattern = %d> %m% +n ); # Initialize logging behaviour Log::Log4perl->init( \$conf ); Log::Log4perl->infiltrate_lwp(); $self->{'logger'} = get_logger(); # create user agent $self->{ua} = LWP::UserAgent->new( parse_head => 0, # No need to parse HTML keep_alive => 1, requests_redirectable => ['POST', 'GET', 'HEAD'] ); $self->{ua}->ssl_opts(verify_hostname => 0, SSL_verify_mode => 0); if ($params{proxy}) { $self->{ua}->proxy(['http', 'https'], $params{proxy}); } else { $self->{ua}->env_proxy(); } $self->{ua}->timeout($self->{timeout}); return $self; } sub request { my ($self, $request, $file) = @_; # $request is a HTTP::Request object, created with only the URL # $file is a message, normally an XML file my $logger = $self->{logger}; my $url = $request->uri(); my $scheme = $url->scheme(); print "\$url = $url\n\t\$scheme = $scheme\n"; print "\t\$self->{ssl_set} = ",$self->{ssl_set},"\n"; print "\t\$self->{ca_cert_dir} = ",$self->{ca_cert_dir},"\n"; print "\t\$self->{ca_cert_file} = ",$self->{ca_cert_file},"\n"; $self->_setSSLOptions() if $scheme eq 'https' && !$self->{ssl_set} +; my $result = HTTP::Response->new( 500 ); eval { $result = $self->{ua}->request($request, $file); }; # check result first if (!$result->is_success()) { # authentication required if ($result->code() == 401) { if ($self->{user} && $self->{password}) { $logger->debug( $log_prefix . "authentication required, submitting credentials" ); # compute authentication parameters my $header = $result->header('www-authenticate'); my ($realm) = $header =~ /^Basic realm="(.*)"/; my $host = $url->host(); my $port = $url->port() || ($scheme eq 'https' ? 443 : 80); $self->{ua}->credentials( "$host:$port", $realm, $self->{user}, $self->{password} ); # replay request eval { if ($OSNAME eq 'MSWin32' && $scheme eq 'https') { alarm $self->{timeout}; } $result = $self->{ua}->request($request, $file); }; if (!$result->is_success()) { $logger->error( $log_prefix . "authentication required, wrong credentials" ); } } else { # abort $logger->error( $log_prefix . "authentication required, no credentials available +" ); } } else { $logger->error( $log_prefix . "communication error: " . $result->status_line() ); } } return $result; } sub _setSSLOptions { my ($self) = @_; # SSL handling if ($self->{no_ssl_check}) { # LWP 6 default behaviour is to check hostname # Fedora also backported this behaviour change in its LWP5 pack +age, so # just checking on LWP version is not enough $self->{ua}->ssl_opts(verify_hostname => 0, SSL_verify_mode => +0) if $self->{ua}->can('ssl_opts'); } else { # only IO::Socket::SSL can perform full server certificate val +idation, # Net::SSL is only able to check certification authority, and +not # certificate hostname IO::Socket::SSL->require(); die "failed to load IO::Socket::SSL, " . "unable to perform SSL certificate validation.\n" . "You can use 'no-ssl-check' option to disable it." if $EVAL_ERROR; # if ($self->{logger}{debug} >= 3) { # $Net::SSLeay::trace = 2; # } print "\t\t\$LWP::VERSION = $LWP::VERSION\n"; if ($LWP::VERSION >= 6) { print "\t\tSetting cert dir and file if available\n"; $self->{ua}->ssl_opts(SSL_ca_file => $self->{ca_cert_file} +) if $self->{ca_cert_file}; $self->{ua}->ssl_opts(SSL_ca_path => $self->{ca_cert_dir}) if $self->{ca_cert_dir}; } } $self->{ssl_set} = 1; } 1;
And here is the test script:
#!/usr/bin/perl use HTTP::Request; use HTTP::Response; use lib './Work'; use REJBlibUA::client; my $method = "POST"; my @requests = (HTTP::Request->new( $method,'https://www.google.ca'), HTTP::Request->new( $method,'https://gremlin.site/cgi- +bin/printenv.pl'), HTTP::Request->new( $method,'https://byerspublishing.com')); my $cnt = 0; foreach my $r (@requests) { my %rp; if ($cnt == 1) { $rp{'ca_cert_file'} = 'client.crt'; $rp{'ca_cert_dir'} = '.'; } my $c = REJBlibUA::client->new(%rp); my $resp = $c->request($r); if ($resp->is_success) { print $resp->decoded_content; } else { print STDERR $resp->status_line, "\n"; } $cnt += 1; }
NB: both client and the gremlin.site server are OpenSuse 13.1 boxes, and byerspublishing.com (a private server, visible only within my LAN for now) is on Ubuntu 12.04 and is not configured to want client side certificates,
Note line 64 in my client package: $self->{ua}->ssl_opts(verify_hostname => 0, SSL_verify_mode => 0); The values of 0 and 0 are the only values that lets my test script get data. If I set only SSL_verify_mode to 1, I get the following error:
2014/07/31 11:42:21> [http client] communication error: 500 SSL negoti +ation failed: 500 SSL negotiation failed:
If I set only verify_hostname to 1, I get the following error:
2014/07/31 11:44:14> [http client] communication error: 500 Can't conn +ect to gremlin.site:443 (Crypt-SSLeay can't verify hostnames) 500 Can't connect to gremlin.site:443 (Crypt-SSLeay can't verify hostn +ames)
Athough above I show only one error in each case, I get the same error in each case for ALL the hosts I try to reach. NB: Setting both of these parameters to 1 gets the same error that verify_hostname produces.
Now, curiously, if I comment out "use Net::SSL (); # From Crypt-SSLeay", the error changes, when both parameters are set to 1.
2014/07/31 11:48:24> [http client] communication error: 500 Can't conn +ect to gremlin.site:443 (certificate verify failed) 500 Can't connect to gremlin.site:443 (certificate verify failed)
I can understand this for byerspublishing.com as it is using only the test certificate provided when one installs Apache2. But this error makes no sense gor gremlin.site, as the ONLY certificates in the relevant directories for Apache are those I created myelf, as I created a rootCA using openssl, and then I used that rootCA to make the certificates both for the server and for the client. So why, in the case of gremlin.site, would there be a problem verifying the certificates? Is the cetificate validation error generated by the server or the client?
I'd appreciate any help you can provide in solving this problem. Any tips for improving client.pm would be appreciated.
Thanks
Ted
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |