A Confession

clerew has asked for the wisdom of the Perl Monks concerning the following question:

Forgive me Father, for I have HACKED.

Well, I suppose that should have been whispered in the ear of the Abbott, but I want the whole company to see it, in the hope that some more experienced Monk can tell me what I should have done.

I have been trying to install Net::SFTP. I suspect that this is an ancient module relying on now-outdated CPAN features, since I have previously used it without problem on a different (Solaris) machine where the available CPAN modules were about 5 years old.

Following advice given a few weeks ago on this forum, I looked for useful modules already compiled for Ubuntu and installed libnet-ssh-perl and libnet-ssh2-perl (and it also seemed to want libgmp3-dev, which is not a Perl module at all).

But it seems that Net::SFTP is not built on top of those things, since it was demanding to see modules for Blowfish, Rsa, Pari and lots more, and attempts to install those lead to inexplicable test failures which required force to get around (I must have called force install ... a dozen times).

Finally, it was just short of Math::Pari, and I was in the ridiculous situation of

cpan[36]> install Math::Pari
Could not read metadata file. Falling back to other methods to determi
+ne prerequisites
Running install for module 'Math::Pari'
Running make for I/IL/ILYAZ/modules/Math-Pari-2.010808.zip
Checksum for /usr/local/CPAN/sources/authors/id/I/IL/ILYAZ/modules/Mat
+h-Pari-2.010808.zip ok

  CPAN.pm: Building I/IL/ILYAZ/modules/Math-Pari-2.010808.zip
yadda
yadda
yadda
make[1]: Entering directory `/usr/local/CPAN/build/Math-Pari-2.010808-
+BwhAPo/libPARI'
make[1]: Leaving directory `/usr/local/CPAN/build/Math-Pari-2.010808-B
+whAPo/libPARI'
make[1]: Entering directory `/usr/local/CPAN/build/Math-Pari-2.010808-
+BwhAPo/libPARI'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/usr/local/CPAN/build/Math-Pari-2.010808-B
+whAPo/libPARI'
Files found in blib/arch: installing files in blib/lib into architectu
+re dependent library tree
Installing /usr/local/lib/perl/lib/perl5/i686-linux-gnu-thread-multi-6
+4int/auto/Math/Pari/Pari.so
Appending installation info to /usr/local/lib/perl/lib/perl5/i686-linu
+x-gnu-thread-multi-64int/perllocal.pod
  ILYAZ/modules/Math-Pari-2.010808.zip
  /usr/bin/make install  -- OK
[download]

So it claimed to have installed it, except that it wasn't there:

cpan[37]> m Math::Pari
Module id = Math::Pari
    CPAN_USERID  ILYAZ (Ilya Zakharevich <cpan@ilyaz.org>)
    CPAN_VERSION 2.010808
    CPAN_FILE    I/IL/ILYAZ/modules/Math-Pari-2.010808.zip
    UPLOAD_DATE  2014-06-01
    MANPAGE      Math::Pari - Perl interface to PARI.
    INST_FILE    (not installed)
[download]

And when I tried to run my application, I got

+ ./transmit-ftp publish.man.ac.uk Web ./newsadmins.html ./committee.h
+tml
Can't locate Math/Pari.pm in @INC (you may need to install the Math::P
+ari module) (@INC contains: /etc/perl /usr/local/lib/perl/5.18.2 /usr
+/local/share/perl/5.18.2 /usr/lib/perl5 /usr/share/perl5 /usr/lib/per
+l/5.18 /usr/share/perl/5.18 /usr/local/lib/site_perl .) at /usr/local
+/lib/perl/5.18.2/Net/SSH/Perl/Util/SSH2MP.pm line 6.
BEGIN failed--compilation aborted at /usr/local/lib/perl/5.18.2/Net/SS
+H/Perl/Util/SSH2MP.pm line 6.
Compilation failed in require at /usr/local/lib/perl/5.18.2/Net/SSH/Pe
+rl/Util.pm line 56.
BEGIN failed--compilation aborted at /usr/local/lib/perl/5.18.2/Net/SF
+TP/Buffer.pm line 9.
Compilation failed in require at /usr/local/lib/perl/5.18.2/Net/SFTP/A
+ttributes.pm line 7.
BEGIN failed--compilation aborted at /usr/local/lib/perl/5.18.2/Net/SF
+TP/Attributes.pm line 7.
Compilation failed in require at /usr/local/lib/perl/5.18.2/Net/SFTP.p
+m line 8.
BEGIN failed--compilation aborted at /usr/local/lib/perl/5.18.2/Net/SF
+TP.pm line 8.
Compilation failed in require at ./transmit-ftp line 3.
BEGIN failed--compilation aborted at ./transmit-ftp line 3.
[download]

And indeed, on looking in /usr/local/lib/perl/5.18.2/Math there was no Pari.pm, but there was a Pari.pm in /usr/local/lib/perl/5.18.2/i686-linux-gnu-thread-multi-64int/Math. So I copied that Pari.pm into where the other one should have been, and Cpan now admitted it was installed:

cpan[38]> m Math::Pari
Module id = Math::Pari
    CPAN_USERID  ILYAZ (Ilya Zakharevich <cpan@ilyaz.org>)
    CPAN_VERSION 2.010808
    CPAN_FILE    I/IL/ILYAZ/modules/Math-Pari-2.010808.zip
    UPLOAD_DATE  2014-06-01
    MANPAGE      Math::Pari - Perl interface to PARI.
    INST_FILE    /usr/local/lib/perl/5.18.2/Math/Pari.pm
    INST_VERSION 2.010808
[download]

So now back to the application:

+ ./transmit-ftp publish.man.ac.uk Web ./newsadmins.html ./committee.h
+tml
Can't locate loadable object for module Math::Pari in @INC (@INC conta
+ins: /etc/perl /usr/local/lib/perl/5.18.2 /usr/local/share/perl/5.18.
+2 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.18 /usr/share/perl/
+5.18 /usr/local/lib/site_perl .) at /usr/local/lib/perl/5.18.2/Net/SS
+H/Perl/Util/SSH2MP.pm line 6.
Compilation failed in require at /usr/local/lib/perl/5.18.2/Net/SSH/Pe
+rl/Util/SSH2MP.pm line 6.
BEGIN failed--compilation aborted at /usr/local/lib/perl/5.18.2/Net/SS
+H/Perl/Util/SSH2MP.pm
.................
[download]

Observe that the error message is now different: "Can't locate loadable object for module Math::Pari in @INC".

Evidently, it is expecting the location of the C-compiled *.so files (which are the bulk of the Math::Pari module) to be visible in the @INC.

So I now perpetrated the HACK, to which I have confessed:

##### HACK ########
push (@INC, "/usr/local/lib/perl/5.18.2/i686-linux-gnu-thread-multi-64
+int");
[download]

(that was actually inside the copies Pari.pm). And my application RAN without problem.

So, Finally, the question for the Monks is "How was I supposed to fix this problem? Or alternatively, should the location of those *.so files have been in @INC all along?"

Comment on A Confession Select or Download Code

Replies are listed 'Best First'.
Re: A Confession by salva (Canon) on Jul 13, 2015 at 06:27 UTC
use Net::SFTP::Foreign instead!	[reply]
Re^2: A Confession by clerew (Novice) on Jul 16, 2015 at 11:17 UTC
Yes. It installed with no problem. Just a few questions: 1. Why did you give it such a strange title? Is "::Foreign" a common usage within CPAN? 2. I used the ::Compat option, and it did not like giving a Debug option to new. 3. The first time I used it (having successfully used Net::SFTP the day before), the upload site refused my connection saying that it had adopted a new public key. Easily fixed in 'known-hosts', but an odd coincidence. I mention is as a 'heads up' on a possible problem. Otherwise, it is working fine.	[reply]
Re^3: A Confession by salva (Canon) on Jul 16, 2015 at 13:36 UTC
Why did you give it such a strange title? Is "::Foreign" a common usage within CPAN? That was not probably my best decision. Anyway, in the beginning Net::SFTP::Foreign was a fork of Net::SFTP that used the OS `ssh` client to connect with the remote host instead of the Net::SSH::Perl module. So, Net::SSH::Perl was the native transport, and `ssh` the foreign one. Later, I rewrote almost all the code, changing its API, for the 1.x series, but the name just stuck. I used the ::Compat option, and it did not like giving a Debug option to new. The right option recognized by Net::SFTP is lowercase `debug`. What happens is that Net::SFTP ignores any option it doesn't recognize while Net::SFTP::Foreign::Compat reports them as errors. I am not sure about Net::SFTP::Foreign aiming at bug-for-bug compatibility... The first time I used it (having successfully used Net::SFTP the day before), the upload site refused my connection saying that it had adopted a new public key. That's how `ssh` handles host authentication. Net::SFTP::Foreign::Compat promises compatibility at the API level, so that you don't have to change your scripts. But under the hood it still let's `ssh` handle the SSH configuration in its way. `N::S::F::Compat` is just a hack to let people with scripts written for Net::SFTP move to Net::SFTP::Foreign easily and in the early days it was also a way to migrate from the 0.x series to 1.x. If you can, you should use the real Net::SFTP::Foreign API that is much more powerful, consistent and tested.	[reply] [d/l] [select]
Re: A Confession by Anonymous Monk on Jul 12, 2015 at 21:15 UTC
So, Finally, the question for the Monks is "How was I supposed to fix this problem? Or alternatively, should the location of those .so files have been in @INC all along?"* It all depends on Configure/Config Here are hints Re^7: Why does PERL5LIB need to be so verbose? (installstyle sitelib installlib vendorlib INST_ARCH INST_VER userelocatableinc @INC) Re^2: How do I tell CPAN to use a local lib? (perllib/perl5lib/perl5opt/-Idirectory/-Mblib/sitecustomize.pl ) Re: Why does PERL5LIB need to be so verbose? ( INSTALL_BASE the local::lib way) Re: cpanm --sudo installs to /root/perl5 instead of /usr	[reply]
Re^2: A Confession by clerew (Novice) on Jul 16, 2015 at 11:36 UTC
Yes, I really ought to set a PERL5LIB Environment Variable, but that is a messy option since every user using my script would have to set PERL5LIB. Well there aren't many users on my machine, but since these scripts are run as cron jobs, root would have to have PERL5LIB set too. The REAL problem is that @INC is hard-coded into the Perl binary, and Ubuntu/Debian have decided to set it to /etc/perl /usr/local/lib/perl/5.18.2 /usr/local/share/perl/5.18.2 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.18 /usr/share/perl/5.18 /usr/local/lib/site_perl . So clearly they expect you to use /usr/local/lib/perl/5.18.2 or /usr/local/share/perl/5.18.2 for your personal libraries, but if your private library includes any C-coded stuff it does not work. Seems like a Ubuntu/Debian Bug to me, and I am minded to report it as such.	[reply]