cpan & strawberry perl errors

BrowserUk has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: cpan & strawberry perl errors by Corion (Patriarch) on Dec 30, 2014 at 14:18 UTC
Your browser has a proxy set, or better network connectivity, than Perl. Inspect your browser network settings closely (likely, the "Network" tab), and configure LWP appropriately. Most likely by setting `$ENV{HTTP_PROXY}` and `$ENV{HTTPS_PROXY}`, preferrably even before your Perl process starts. Especially for HTTPS, OpenSSL / SSLeay like to have first grab at these variables.	[reply] [d/l] [select]
Re^2: cpan & strawberry perl errors by BrowserUk (Patriarch) on Dec 30, 2014 at 14:51 UTC
As you can see, the LWP failing url works fine in my browser and it has no proxies configured. Nor do I have any proxies set in my environment: `C:\test>set proc PROCESSOR_ARCHITECTURE=x86 PROCESSOR_ARCHITEW6432=AMD64 PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 15 Stepping 11, GenuineInt +el PROCESSOR_LEVEL=6 PROCESSOR_REVISION=0f0b C:\test>set http Environment variable http not defined` [download] The weird thing is that the same urls that fail (instantly and multiple times) will subsequently succeed occasionally. But in my browser, they always succeed. With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice.	[reply] [d/l]
Re^3: cpan & strawberry perl errors by Corion (Patriarch) on Dec 30, 2014 at 15:12 UTC
I would use WireShark to find out what is happening on the network layer. As your browser knows how to connect, but Perl doesn't, something in their behaviour is different. Maybe using WireShark, you can find out how their behaviour differs. I think you've mentioned using Opera, and I also remember that Opera (Mini) has a Turbo Mode, where it uses an Opera-operated proxy. This might not show up in your general proxy settings. Also consider whether Opera gets a different IP address for `search.cpan.org` than what Perl gets - Perl might end up with an unavailable mirror (sometimes) while Opera might be lucky due to longer DNS caching within the browser (instead of relying on the OS to provide "current" DNS).	[reply] [d/l]
Re^4: cpan & strawberry perl errors by BrowserUk (Patriarch) on Dec 30, 2014 at 18:43 UTC
Re: cpan & strawberry perl errors by syphilis (Archbishop) on Dec 31, 2014 at 00:06 UTC
Fetching with LWP: http://cpan.strawberryperl.com/authors/01mailrc.txt.gz My strawberry downloads without issue, but from a different location: `Fetching with LWP: http://www.cpan.org/authors/01mailrc.txt.gz` [download] I can configure this by changing the `urllist` setting in /Users/me/.cpan/CPAN/MyConfig.pm I'd try setting it to http://www.cpan.org and see if that makes any difference. It may make no difference at all, but it shouldn't take long to test. Cheers, Rob	[reply] [d/l] [select]
Re^2: cpan & strawberry perl errors by BrowserUk (Patriarch) on Dec 31, 2014 at 00:26 UTC
Taking out cpan.strawberryperl.com from the urllist avoids the redirect, but still the LWP get from cpan.pair.com fails with status500; but the fallback to wget succeeds: C:\test>cpan cpan shell -- CPAN exploration and modules installation (v2.05) Enter 'h' for help. cpan> o conf urllist urllist 0 [http://cpan.strawberryperl.com/] 1 [http://www.cpan.org/] Type 'o conf' to view all configuration items cpan> o conf urllist shift commit: wrote 'C:\perl5.18\perl\lib/CPAN/Config.pm' cpan> o conf urllist urllist 0 [http://www.cpan.org/] Type 'o conf' to view all configuration items cpan> reload index Fetching with LWP: http://www.cpan.org/authors/01mailrc.txt.gz LWP failed with code[500] message[Can't connect to www.cpan.org:80] Trying with c:/perl64/bin/wget.exe -O "C:\Perl5.18\cpan\sources\authors\01mail +rc.txt.gz.tmp7328" to get http://www.cpan.org/authors/01mailrc.txt.gz --2014-12-31 00:22:29-- http://www.cpan.org/authors/01mailrc.txt.gz Resolving www.cpan.org... 94.242.223.198 Connecting to www.cpan.org\|94.242.223.198\|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 228720 (223K) [application/x-gzip] Saving to: `C:/Perl5.18/cpan/sources/authors/01mailrc.txt.gz.tmp7328' 100%[===============================================================>] + 228,720 --.-K/s in 0.1s 2014-12-31 00:22:30 (1.56 MB/s) - `C:/Perl5.18/cpan/sources/authors/01 +mailrc.txt.gz.tmp7328' saved [228720/228720] Fetching with LWP: http://www.cpan.org/modules/02packages.details.txt.gz LWP failed with code[500] message[Can't connect to www.cpan.org:80] Trying with c:/perl64/bin/wget.exe -O "C:\Perl5.18\cpan\sources\modules\02pack +ages.details.txt.gz.tmp7328" to get http://www.cpan.org/modules/02packages.details.txt.gz --2014-12-31 00:22:30-- http://www.cpan.org/modules/02packages.detail +s.txt.gz Resolving www.cpan.org... 94.242.223.198 Connecting to www.cpan.org\|94.242.223.198\|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 1615484 (1.5M) [application/x-gzip] Saving to: `C:/Perl5.18/cpan/sources/modules/02packages.details.txt.gz +.tmp7328' 100%[===============================================================>] + 1,615,484 2.11M/s in 0.7s 2014-12-31 00:22:31 (2.11 MB/s) - `C:/Perl5.18/cpan/sources/modules/02 +packages.details.txt.gz.tmp7328' saved [1615484/1615484] Fetching with LWP: http://www.cpan.org/modules/03modlist.data.gz LWP failed with code[500] message[Can't connect to www.cpan.org:80] Trying with c:/perl64/bin/wget.exe -O "C:\Perl5.18\cpan\sources\modules\03modl +ist.data.gz.tmp7328" to get http://www.cpan.org/modules/03modlist.data.gz --2014-12-31 00:22:31-- http://www.cpan.org/modules/03modlist.data.gz Resolving www.cpan.org... 94.242.223.198 Connecting to www.cpan.org\|94.242.223.198\|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 248 [application/x-gzip] Saving to: `C:/Perl5.18/cpan/sources/modules/03modlist.data.gz.tmp7328 +' 100%[===============================================================>] + 248 --.-K/s in 0s 2014-12-31 00:22:31 (2.55 MB/s) - `C:/Perl5.18/cpan/sources/modules/03 +modlist.data.gz.tmp7328' saved [248/248] Database was generated on Wed, 31 Dec 2014 00:20:56 GMT Updating database file ... [download] With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice.	[reply] [d/l]
Re: cpan & strawberry perl errors by Anonymous Monk on Dec 31, 2014 at 08:53 UTC
Basically, why would LWP fail repeatedly from many sources most of the time but occassionally succeed? What do you get if you try `perl -S -MDevel::VersionDump lwp-request -UuSsEd url` on some of those urls?	[reply]
Re^2: cpan & strawberry perl errors by BrowserUk (Patriarch) on Dec 31, 2014 at 10:34 UTC
Notice that with LWP the redirect to pair.com fails with 500; but succeeds with wget. It looks for all the world like pair.com is rejecting LWP's useragent string? C:\test>perl -S -MDevel::VersionDump lwp-request -Ed http://cpan.straw +berryperl.com/authors/01mailrc.txt.gz Perl version: v5.18.4 on MSWin32 (C:\Perl5.18\perl\bin\perl.exe) Carp - 1.3301 Config - 5.018004 Config_git.pl - Unknown Config_heavy.pl - Unknown Devel::VersionDump - 0.02 DynaLoader - 1.18 Encode - 2.62 Encode::Alias - 2.18 Encode::Byte - 2.04 Encode::Config - 2.05 Encode::Encoding - 2.07 Encode::Locale - 1.03 Exporter - 5.70 Exporter::Heavy - 5.70 File::Basename - 2.84 Getopt::Long - 2.42 HTTP::Date - 6.02 HTTP::Status - 6.03 List::Util - 1.41 Scalar::Util - 1.41 Time::Local - 1.2300 URI - 1.64 URI::Escape - 3.31 URI::Heuristic - 4.20 Win32::API - 0.79 Win32::API::Struct - 0.65 Win32::API::Type - 0.69 XSLoader - 0.16 base - 2.18 bytes - 1.04 constant - 1.27 overload - 1.22 overloading - 0.02 strict - 1.07 vars - 1.03 warnings - 1.18 warnings::register - 1.02 GET http://cpan.strawberryperl.com/authors/01mailrc.txt.gz User-Agent: lwp-request/6.03 libwww-perl/6.08 302 Found Connection: close Date: Wed, 31 Dec 2014 10:30:00 GMT Location: http://cpan.pair.com/authors/01mailrc.txt.gz Server: Apache Vary: Accept-Encoding Content-Length: 228 Content-Type: text/html; charset=iso-8859-1 Client-Date: Wed, 31 Dec 2014 10:30:22 GMT Client-Peer: 69.163.152.144:80 Client-Response-Num: 1 Title: 302 Found GET http://cpan.pair.com/authors/01mailrc.txt.gz User-Agent: lwp-request/6.03 libwww-perl/6.08 500 Can't connect to cpan.pair.com:80 Content-Type: text/plain Client-Date: Wed, 31 Dec 2014 10:30:22 GMT Client-Warning: Internal response C:\test>wget http://cpan.pair.com/authors/01mailrc.txt.gz --2014-12-31 10:31:47-- http://cpan.pair.com/authors/01mailrc.txt.gz Resolving cpan.pair.com... 66.39.76.93 Connecting to cpan.pair.com\|66.39.76.93\|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 228720 (223K) [application/x-gzip] Saving to: `01mailrc.txt.gz' 100%[=====================>] 228,720 240K/s in 0.9s 2014-12-31 10:31:49 (240 KB/s) - `01mailrc.txt.gz' saved [228720/22872 +0] C:\test> [download] With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice.	[reply] [d/l]
Re^3: cpan & strawberry perl errors by Anonymous Monk on Dec 31, 2014 at 10:55 UTC
d'oh, that doesnt give LWP version info ... seems to work here just fine $ perl -S -MDevel::ModuleDumper lwp-download http://cpan.strawberryper +l.com/authors/01mailrc.txt.gz Saving to '01mailrc.txt.gz'... 223 KB received in 0 seconds (288 KB/sec) Perl 5.016001 Compress::Raw::Zlib 2.063 Encode 2.55 Encode::Alias 2.18 Encode::Byte 2.04 Encode::Config 2.05 Encode::Encoding 2.07 Encode::Locale 1.03 Errno 1.15 Fcntl 1.11 File::Basename 2.84 File::Glob 1.17 File::GlobMapper 1.000 Getopt::Std 1.07 HTML::Entities 3.69 HTML::HeadParser 3.69 HTML::Parser 3.7 HTTP::Config 6.00 HTTP::Date 6.02 HTTP::Headers 6.05 HTTP::Message 6.06 HTTP::Request 6.00 HTTP::Response 6.04 HTTP::Status 6.03 IO 1.25_06 IO::Compress::Base::Common 2.063 IO::Compress::Gzip::Constants 2.063 IO::Compress::Zlib::Extra 2.063 IO::File 1.16 IO::Handle 1.33 IO::Seekable 1.1 IO::Socket 1.34 IO::Socket::INET 1.33 IO::Socket::IP 0.31 IO::Socket::UNIX 1.24 IO::Uncompress::Adapter::Inflate 2.063 IO::Uncompress::Base 2.063 IO::Uncompress::Gunzip 2.063 IO::Uncompress::RawInflate 2.063 List::Util 1.33 LWP 6.08 LWP::ConnCache 6.02 LWP::MediaTypes 6.02 LWP::MemberMixin none LWP::Protocol 6.06 LWP::Protocol::http none LWP::UserAgent 6.06 Net::HTTP 6.07 Net::HTTP::Methods 6.07 POSIX 1.30 Scalar::Util 1.33 SelectSaver 1.02 Socket 2.006 Storable 2.39 Symbol 1.07 Tie::Hash 1.04 Time::Local 1.2300 URI 1.60 URI::Escape 3.31 URI::http none URI::_generic none URI::_idna none URI::_punycode 0.04 URI::_query none URI::_server none Win32::API 0.75 Win32::API::Struct 0.64 Win32::API::Type 0.67 Win32::Console 0.09 [download]	[reply] [d/l]
Re^4: cpan & strawberry perl errors by BrowserUk (Patriarch) on Dec 31, 2014 at 11:05 UTC
Re^5: cpan & strawberry perl errors by Anonymous Monk on Dec 31, 2014 at 11:13 UTC
Re^4: cpan & strawberry perl errors by BrowserUk (Patriarch) on Dec 31, 2014 at 11:08 UTC
Re^5: cpan & strawberry perl errors by Anonymous Monk on Dec 31, 2014 at 11:18 UTC
Some notes below your chosen depth have not been shown here