comment on

Dear Monks, I am accessing a server with LWP::UserAgent over SSL for some time now and just recently I encountered this 500 Server closed connection without sending any data back (Client-Warning: Internal response). It appears randomly but often. Some comments I read suggested that this happens at the SSL setup phase where server closes connection and LWP issues this warning (hence Internal response). I have allowed a large timeout of 600s in the setup of LWP::UserAgent.

I have created a simple script to hit an irrelevant server (not the one I use for obvious reasons) just to test whether I have setup LWP+SSL correctly and also getting maximum debugging information.

#!/usr/bin/perl

use Test::More;

use LWP::ConsoleLogger::Easy qw( debug_ua );
use IO::Socket::SSL;
use LWP::UserAgent;

my $DEBUG = 1;

my $num_tests = 0;

my %pages = (
        # some name for the page to hit
        'google' =>
                [
                        # the url to hit
                        'https://www.google.com',
                        # regex obj to validate returned content
                        qr/Google Search/,
                ],
);

my $ua = LWP::UserAgent->new(ssl_opts => { verify_hostname => 1 });
if( $DEBUG == 1 ){
        LWP::ConsoleLogger::Easy::debug_ua($ua, 6);
        $IO::Socket::SSL::DEBUG = 3;
}

my ($response, $content, $aurl, $regex_validator, $apage);

foreach $apage (keys %pages){
        ($aurl, $regex_validator) = @{$pages{$apage}};
        $response = $ua->get($aurl);
        ok(defined($response), "$apage hit : $aurl"); $num_tests++;
        ok($response->is_success==1, "$apage hit got success status co
+de : $aurl") or BAIL_OUT("$apage : failed to hit : $aurl"); $num_test
+s++;
        $content = $response->decoded_content;
        ok($content =~ $regex_validator, "$apage validated OK") or pri
+nt "$apage : failed to validate the following content:\n".$content."\
+nend content.\n"; $num_tests++;
}
# END
done_testing($num_tests);
[download]

Here is part of the output:

DEBUG: .../IO/Socket/SSL.pm:2853: new ctx 93957261363856
DEBUG: .../IO/Socket/SSL.pm:692: socket not yet connected
DEBUG: .../IO/Socket/SSL.pm:694: socket connected
DEBUG: .../IO/Socket/SSL.pm:717: ssl handshake not started
DEBUG: .../IO/Socket/SSL.pm:750: using SNI with hostname www.google.co
+m
DEBUG: .../IO/Socket/SSL.pm:785: request OCSP stapling
DEBUG: .../IO/Socket/SSL.pm:806: set socket to non-blocking to enforce
+ timeout=180
DEBUG: .../IO/Socket/SSL.pm:819: call Net::SSLeay::connect
DEBUG: .../IO/Socket/SSL.pm:822: done Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:832: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:842: waiting for fd to become ready: SSL w
+ants a read first
DEBUG: .../IO/Socket/SSL.pm:862: socket ready, retrying connect
DEBUG: .../IO/Socket/SSL.pm:819: call Net::SSLeay::connect
DEBUG: .../IO/Socket/SSL.pm:2707: ok=1 [2] /OU=GlobalSign Root CA - R2
+/O=GlobalSign/CN=GlobalSign/OU=GlobalSign Root CA - R2/O=GlobalSign/C
+N=GlobalSign
[download]

When hitting my server (and not google) the process hangs exactly at waiting for fd to become ready: SSL wants a read first but with a slightly different output (it does not have did not get stapled OCSP response) and repeated instances of:

DEBUG: .../IO/Socket/SSL.pm:832: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:842: waiting for fd to become ready: SSL w
+ants a read first
DEBUG: .../IO/Socket/SSL.pm:862: socket ready, retrying connect
DEBUG: .../IO/Socket/SSL.pm:819: call Net::SSLeay::connect
DEBUG: .../IO/Socket/SSL.pm:822: done Net::SSLeay::connect -> -1
[download]

I am asking for your wisdom. My questions are:

1) is the way I am using LWP+SSL in the example script correct? Any suggestions on best/preferred way to do this?
2) Does the output done Net::SSLeay::connect -> -1 signify an error? I could not figure out what this -1 is (as opposed to 1 in other instances) from documentation.
3) Any suggestions on improving debug/trace levels?

Run on latest Linux kernel with perl v5.26.2, OpenSSL 1.1.0, IO::Socket::SSL 2.060, Net::SSLeay 1.85

In reply to Net::SSLeay::connect -> -1 by bliako

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		`&`
	<		`<`
	>		`>`
	[		`[`
	]		`]`

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.