There is no Crypt::SSL. It might be that you are referring to Crypt::SSLeay which was long time ago used to provide the SSL support in LWP. But since more than 8 years ago this is now done with IO::Socket::SSL + Net::SSLeay instead.

Hi @localshop, thanks for taking an interest.

• I am using strict & warnings.
• The failing URLs are consistent, though I can't figure out any sort of pattern in either the URLs or the content they ought to download.
• I am not sure if the server sets a MIME type at all. If it does it will be some sort of catch all for binary files like application/octet-stream
• I have tried with Internet Explorer, Chrome & Firefox. All work fine. I have not tried with any command line tools or other perl libraries.
• The LWP client is running on Windows (Versions 7, 8, 10 & server 2008) as a Virtual Machine running on Linux KVM virtualiser. Hosted on Ubuntu 14.04. The server is in a Docker container under CentOS & Rancher 1.5x.
• LWP requests are in the clear on HTTP port 80.
• Not sure about LWP versions. Will that be built in to the Perl distro, or separately installed?
• I have confirmed this behaviour by installing later versions of perl on top of each other until the code starts working.

The overall application is an internal file storage system. The client is fairly simple, and is mostly concerned with supplying authentication credentials, checking the SHA1 checksum of the file as it is downloaded, and re-trying failed downloads. The server checks credentials, records the fact that the client downloaded the file in a database, and finds the file in one of a set of caches, local and remote file servers.

I can't upload a code example at the moment because I am not at work, but I will create and upload a minimal sample when I get a chance.

Here is a code fragement, scrubed of all the company internal stuff:

sub getfile {
    my ( $url, $outfile, $options ) = @_;
    $options ||= {};

    # Default option values
    my $max_retries = $options->{'max_retries'} || 5;
    my $file_host   = $options->{'file_host'}   || $ENV{FILE_HOST} || 
+"file-host.companyname.internal";
    my $ua          = $options->{'ua'}          || LWP::UserAgent->new
+( keep_alive => 1, cookie_jar => {} );

    # Force to a positive integer.
    $max_retries = int($max_retries);
    $max_retries = 1 if $max_retries < 1;

    # Construct auth headers
    my %auth_headers = ();
    if( $options->{'basic_auth_creds'} )
    {
        %auth_headers = ( 'Authorization' => "Basic ".encode_base64($o
+ptions->{'basic_auth_creds'}) );
    }

    my ( $csum_type, $csum ) = csum_to_lookup($url);

    if ( $csum_type eq 'sha1' ) {
        $url         = "http://$file_host/getfile/" . $csum;
        $wanted_sha1 = $csum;
    }
    else {
        ...        # Look in the database to convert to SHA1
    }

    my $result;
ATTEMPT: foreach my $attempt ( 1 .. $max_retries ) {
        debug("attempt $attempt to get url: '$url'");

        my $f_out;

        # This will contain the sha1 of the downloaded data once the d
+ownload is complete
        my $digest = Digest->new('SHA-1');

        # NB: This uses the request method on LWP::UA, that takes an i
+nstance of HTTP::Request and a callback function.
        # See: https://metacpan.org/pod/LWP::UserAgent#ua-request-requ
+est-content_cb
        $result = $ua->request(
            GET($url, %auth_headers),    # Func exported from HTTP::Re
+quest::Common, returns an instance of HTTP::Request
            sub {
                my ( $data, $res ) = @_;
                if ( $res->is_success ) {
                    unless ($f_out) {
                        $f_out = open_outfile( $outfile, $nooverwrite 
+) or croak "unable to open $outfile\n";
                    }
                    print $f_out $data;
                    $digest->add($data) if defined $wanted_sha1;
                }
            },
        );

        unless( $result->request->uri->eq($url) )
        {
            print "$url has redirected to ".$result->request->uri."\n"
+;
        }

        # close file or flush the filehandle to disk
        # FIXME close or flush could fail (eg if insufficient disk spa
+ce). this should be bubbled up
        if (ref($outfile)) {
            $outfile->flush();
        }
        else {
            close $f_out if defined($f_out);
        }

        # For error messages if something goes wrong.
        my $resp_string = sprintf "%d - %s", $result->code, $result->m
+essage;

        # Check the downloaded data is what we expect
        # (Unless we have no checksum when downloading an arbitrary UR
+L)
        # TODO: this segment checks sha1 even if download fails - unne
+cessary warning!
        if ( defined $wanted_sha1 )
        {
            my $download_failed = 0;

            if( ! -f $outfile || 0 == -s $outfile )
            {
                unlink $outfile unless ref($outfile);
                warn "http downloaded failed ($resp_string): Wanted SH
+A1:$wanted_sha1 but got an empty file";
                $download_failed = 1;

            }
            elsif( $digest->hexdigest ne $wanted_sha1 ) {
                unlink $outfile unless ref($outfile);
                warn "http downloaded failed ($resp_string): Wanted SH
+A1:$wanted_sha1 but got:" . $digest->hexdigest;
                $download_failed = 1;
            }

            if( $download_failed ) {
                if( $attempt == $max_retries ) {
                    my $download_url = $result->request->uri;
                    my $hostname = $result->request->uri->host;
                    my $timestamp = scalar gmtime();
                    if( my $packed_ip = gethostbyname( $hostname  ) )
                    {
                        my $file_server_ip = inet_ntoa($packed_ip);   
+ # Using old style pure perl instead of a modern library
                        warn "$timestamp : Request was to $download_ur
+l on IP: $file_server_ip";
                    }
                    else
                    {
                        warn "$timestamp : Request was to $download_ur
+l but cannot resolve IP address for $hostname";
                    }
                }
                else
                {
                    next ATTEMPT;
                }
            }
        }

        if ( $result->is_success ) {
            last ATTEMPT;
        }
        else {
            unlink $outfile unless ref($outfile);
            carp "$url could not be retrieved - $resp_string\n";

            # Delay before the next attempt
            sleep 2**$attempt
                unless $attempt == $max_retries;
        }
    }

    return wantarray() ? ( $result->is_success(), $result ) : $result-
+>is_success();
}
[download]

FWIW, I think the most likely culprit will be a change in the version(s) of some module(s) between 5.24.0 and 5.24.1 - as already suggested by Anonymous Monk and localshop.

With Strawberry Perl the LWP version is at 6.15 in both 5.24.0 and 5.24.1, so we should be able to rule it out.
However, IO::Socket::SSL was bumped from 2.027 in Strawberry-5.24.0 to 2.043 in Strawberry-5.24.1.
And Net::SSLeay was bumped from 1.74 to 1.80.

I'd be inclined to update those 2 modules in Strawberry-5.24.0 to the version used in 5.24.1 and see if that fixes the issue.

Of course, there may be other relevant modules that were upgraded from 5.24.0 to 5.24.1, in which case you can continue on with the procedure if the Net::SSLeay and IO::Socket::SSL upgrades don't fix the issue ;-)

Cheers,
Rob