Re^2: Why is the CPAN shell not DWIMming?

Intrepid wrote:

The long delays are due to the failures of LWP::UserAgent and Net::FTP. CPAN will continue to try those approaches to retrieving the needed files from the mirror host until the lengthy failure timeouts are reached, then will (as reported) fall back to a less sophisticated means.

Let me provide some more data that may provide experienced monks with a clue to the solution.

Once I solved the permissions problem (see other posting), I tackled the question of the mirror I was using. I studied the documentation in CPAN.pm and learned that I could define preferences by making changes in $HOME/.cpan/CPAN/MyConfig.pm. I discovered that I had no such file, perhaps because I had never attempted to make configuration changes since installing 5.8.4. So I copied over /usr/local/lib/perl5/5.8.4/CPAN/Config.pm and, after some trial and error, added the following values:

  'urllist' => [ qw|
      ftp://mirror.cc.columbia.edu/pub/software/cpan/
      ftp://ftp.duke.edu/pub/perl/
      ftp://cpan.cse.msu.edu/
  | ],
[download]

I'd used all three of these mirrors previously. I then decided to test this configuration by installing a module with relatively few prerequisites and no .XS: brian d foy's Test::Data. To my chagrin, I experienced the same type of delays I'd experienced with ftp.perl.org.

Fetching with LWP:
  ftp://mirror.cc.columbia.edu/pub/software/cpan/authors/id/B/BD/BDFOY
+/Test-Data-1.17.tar.gz
LWP failed with code[404] message[File 'Test-Data-1.17.tar.gz' not fou
+nd]
Fetching with Net::FTP:
  ftp://mirror.cc.columbia.edu/pub/software/cpan/authors/id/B/BD/BDFOY
+/Test-Data-1.17.tar.gz
Couldn't fetch Test-Data-1.17.tar.gz from mirror.cc.columbia.edu
Fetching with LWP:
  ftp://ftp.duke.edu/pub/perl/authors/id/B/BD/BDFOY/Test-Data-1.17.tar
+.gz
 LWP failed with code[400] message[FTP return code 000]
Fetching with Net::FTP:
  ftp://ftp.duke.edu/pub/perl/authors/id/B/BD/BDFOY/Test-Data-1.17.tar
+.gz
Couldn't fetch Test-Data-1.17.tar.gz from ftp.duke.edu
Fetching with LWP:
  ftp://cpan.cse.msu.edu/authors/id/B/BD/BDFOY/Test-Data-1.17.tar.gz
LWP failed with code[400] message[FTP return code 000]
Fetching with Net::FTP:
  ftp://cpan.cse.msu.edu/authors/id/B/BD/BDFOY/Test-Data-1.17.tar.gz
Couldn't fetch Test-Data-1.17.tar.gz from cpan.cse.msu.edu
[download]

Only after failing with both LWP::UserAgent and Net::FTP on each of the three mirrors did CPAN.pm finally do the right thing:

Issuing "/usr/bin/ftp -n"
Connected to shadow.cc.columbia.edu.
220-            WELCOME to the FTP archives at mirror.cc.columbia.edu
...
[download]

500 Unknown command.
227 Entering Passive Mode (128,59,59,147,87,161)
150 Opening BINARY mode data connection for Test-Data-1.17.tar.gz (101
+44 bytes).
100% |*************************************| 10144      41.33 KB/s    
+00:00 ETA
226 File send OK.
10144 bytes received in 00:00 (40.76 KB/s)
221 Goodbye.
...
Writing /usr/local/lib/perl5/site_perl/5.8.4/darwin-2level/auto/Test/D
+ata/.packlist
Appending installation info to /usr/local/lib/perl5/5.8.4/darwin-2leve
+l/perllocal.pod
  /usr/bin/make install  -- OK
[download]

... at which point the installation went like a bat outta hell. This is consistent with what Intrepid reported he has seen.

The one new datum I'd like to add is this: I experience similarly long delays when using ftp on my iBook -- but I don't when using it on my Linux or Windows boxes. Suppose I'm ftp-ing to my own web space:

ftp> cd Storage
250 CWD command successful.
ftp> ls
229 Entering Extended Passive Mode (|||57117|)
500 Illegal EPRT command
200 PORT command successful
[download]

I know from experience that at ls or dir, the service will hang and in all likelihood time out -- as it did just now as I was testing it. This was the outcome:

421 Service not available, remote server timed out. Connection closed
[download]

It's the Extended Passive Mode I'm suspicious of. I don't get that on my other boxes; I get plain Passive Mode on those.

I posted this problem on the ny.pm mailing list earlier this year, and Ben Holtzman turned me on to the program lftp written by a Russian hacker and available (I think) on Sourceforge. I have no problem using lftp (or, for that matter, the Mac ftp client Fetch) ... except when I have a script that uses Net::FTP.

So I'm wondering if the problem is not with ftp.perl.org but with the version of ftp that comes with Mac OS X. But if that's the case, why did /usr/bin/ftp -n succeed where LWP::UserAgent and Net::FTP failed?

It just gets more mysterious!

jimk

Comment on Re^2: Why is the CPAN shell not DWIMming? Select or Download Code