in reply to Re^2: LWP::Simple::get($url) does not work for particular urls (SSL certificate verification)
in thread LWP::Simple::get($url) does not work for particular urls

If there was issue of SSL certificate then why i am directly able to see the data in web browser when i browse the link https://www.cryptopia.co.nz/api/GetCurrencies in the web browser?

Hi, this is because your desktop browser uses the Certificate Authority root certificates that are distributed with your operating system, whereas LWP::UserAgent does not know about them. You can force LWP to use the same root certificates by setting the value of the environment variable HTTPS_CA_DIR to the location on your system.

On my system the certificate directory is /etc/ssl/certs.

Without:

use strict; use warnings; use LWP::UserAgent; use JSON; use Data::Dumper; my $url = "https://www.cryptopia.co.nz/api/GetCurrencies"; my $res = LWP::UserAgent->new->get( $url ); die $res->content if not $res->is_success; print Dumper decode_json $res->decoded_content; __END__
Output:
$ perl 1206484.pl | head -20 Can't connect to www.cryptopia.co.nz:443 (certificate verify failed) SSL connect attempt failed error:14090086:SSL routines:ssl3_get_server +_certificate:certificate verify failed at /home/nick/perl5/perlbrew/p +erls/perl-5.26.1/lib/site_perl/5.26.1/LWP/Protocol/http.pm line 46.
Now with the UA pointed at the CA cert store:
use strict; use warnings; use LWP::UserAgent; use JSON; use Data::Dumper; BEGIN { $ENV{'HTTPS_CA_DIR'} = '/etc/ssl/certs'; } my $url = "https://www.cryptopia.co.nz/api/GetCurrencies"; my $res = LWP::UserAgent->new->get( $url ); die $res->content if not $res->is_success; print Dumper decode_json $res->decoded_content; __END__
Output:
$ perl 1206484.pl | head -20 $VAR1 = { 'Message' => undef, 'Success' => bless( do{\(my $o = 1)}, 'JSON::PP::Boolean' ), 'Error' => undef, 'Data' => [ { 'MaxWithdraw' => '90000000', 'IsTipEnabled' => bless( do{\(my $o = 0)}, 'JS +ON::PP::Boolean' ), 'DepositConfirmations' => 20, 'Status' => 'OK', 'MinWithdraw' => '20000', 'Name' => '1337', 'Algorithm' => 'POS', 'Symbol' => '1337', 'MinBaseTrade' => '2e-05', 'StatusMessage' => undef, 'MinTip' => '166.66666666', 'ListingStatus' => 'Active', 'Id' => 331, 'WithdrawFee' => '0.01'

Hope this helps!


The way forward always starts with a minimal test.
  • Comment on Re^3: LWP::Simple::get($url) does not work for particular urls (SSL certificate verification)
  • Select or Download Code

Replies are listed 'Best First'.
Re^4: LWP::Simple::get($url) does not work for particular urls (SSL certificate verification)
by noxxi (Pilgrim) on Dec 31, 2017 at 20:51 UTC
    > ... this is because your desktop browser uses the Certificate Authority root certificates that are distributed with your operating system, whereas LWP::UserAgent does not know about them.

    The problem is not that the root CA is not trusted by LWP but that the intermediate CA is missing (broken server setup) and thus the trust chain cannot be created. Desktop browsers often successfully work around such broken setups by using cached certificates or downloading missing CA from the web but other clients (Python, Perl, Java, mobile apps...) don't do this workarounds and thus fail with broken sites.

      Thanks for the more accurate information, I defer to an expert, which I am not!


      The way forward always starts with a minimal test.
Re^4: LWP::Simple::get($url) does not work for particular urls (SSL certificate verification)
by ssara (Acolyte) on Dec 31, 2017 at 17:03 UTC

    Thanks a lot for the information.