SSH2 scp_get problem

RaduH has asked for the wisdom of the Perl Monks concerning the following question:

Hello,
The code snippet I have a question about is this one:

my $ssh2 = Net::SSH2->new();
$ssh2->connect($sourceServer) or die ("Unable to connect"); 
$ssh2->auth_keyboard('user', 'password') or die ("Unable to login"); 
$ssh2->scp_get($sourceFileName);
$ssh2->disconnect();
[download]

I am using Net::SSH2. The problem is that the scp_get call hangs indefinitely. I am trying to download a 15MB file. It seems it gets stuck after about 12-13MB. Interesting is that if I repeat the experiment, it seems to stop at the same file size, for a while, then it seems to stick to that new value for another while, and so on. Anyway, it doesn't make any progress past that point (not even a crash with or without an error message).

I have the similar scp_put call that uploads that file and that upload works like a charm every time. Not a single problem. On the flip side, the download call I pasted did not work even once. I'd appreciate some help to get past this problem, if anybody knows the way out.

Thanks,
Radu

Comment on SSH2 scp_get problem Download Code

Replies are listed 'Best First'.
Re: SSH2 scp_get problem by Krambambuli (Curate) on Oct 17, 2007 at 22:46 UTC
I can reproduce the problem now on my WS. If you activate debug via $ssh2->debug(1); you'll see that things are going wrong as soon as an 'incomplete' block is read, i.e. 8192 is expected and less than that is actually read. Once this happens, the transfer will not recover any more. I've played a bit with the code, without success so far; it seems to me that the real problem is in the underlying libssl2, but I'm not sure yet. Update: I can confirm now that it is almost sure a libssh2 problem. The library version I have here is version 0.17. I've started looking through the code, but I'm unsure I'll be able to offer a patch anytime soon. One other thing to try would be to test what happens with a different library version. Update once more: I've recompiled libssh2 and rebuild Net::SSH2 for libssh2 version 0.16, 0.15, and the problem persisted. Then I tried libssh2 version 0.12 - and that was a success, I could repeatedly scp a big file (~26MB) without any problem. Success also with version 0.14. Now that seems to imply that actually Net::SSH2 (working ok with libssh2 version 0.14, failing with version >= 0.15) is behind and would need an update. What exactly that would be, I do not know. Yet ;)	[reply]
Re^2: SSH2 scp_get problem by RaduH (Scribe) on Oct 18, 2007 at 20:24 UTC
yup, my libssh2 is also 0.17 :( I'm also in touch with David Robins (the developer of the module)... like you observed, SCP works for me too (all the time).	[reply]
Re: SSH2 scp_get problem by Krambambuli (Curate) on Oct 17, 2007 at 20:05 UTC
Does the same get transfer succeed without a hickup if you're doing it by hand, with a command-line scp ? The answer to this question might help in understanding if it's more a network problem or indeed a Perl problem.	[reply]
Re^2: SSH2 scp_get problem by RaduH (Scribe) on Oct 17, 2007 at 20:30 UTC
I tried both a scp and a sftp session and they both work without any problems. Thanks, Radu	[reply]
Re^3: SSH2 scp_get problem by RaduH (Scribe) on Oct 17, 2007 at 20:51 UTC
Ah, one more thing... the transfer is between two accounts on the same Linux machine so there's traffic up and down the network software stack but no real network traffic between two hosts... in case it makes a difference. Radu	[reply]
Re^4: SSH2 scp_get problem by RaduH (Scribe) on Oct 17, 2007 at 20:57 UTC
Re^4: SSH2 scp_get problem by Krambambuli (Curate) on Oct 17, 2007 at 21:07 UTC
Re^5: SSH2 scp_get problem by RaduH (Scribe) on Oct 17, 2007 at 22:00 UTC
Re: SSH2 scp_get problem by Anonymous Monk on Oct 17, 2007 at 22:42 UTC
Hi Just a guess, is there a 'Timeout' value that could be set higher? We had a similar problem with FTP that we solved in this way. J.C.	[reply]
Re^2: SSH2 scp_get problem by RaduH (Scribe) on Oct 18, 2007 at 15:11 UTC
Well, I could buy the timeout story if it happened somewhat later. The transfer is between two accounts on the same machine. It's almost instantaneous... If this is indeed a timeout issue and it manifests itself like this when both ends are on the same machine, I suspect it makes the package useless for use to transfer files between machines (home computer with dialup connection...? I'm sure there are plenty out there)... Well, just a thought.. I also got in touch with the module developer and he'll take a look once I provide him with more information he asked for.	[reply]
Re: SSH2 scp_get problem by Robot (Novice) on Oct 19, 2007 at 13:11 UTC
Hi, I remember that I also did have huge problems with Net-SSH stuff (of course, there were no problems when using the system binaries). Basically, my suggestion for you is to use Net::SFTP for file transfers and Expect stuff for direct invocation and control of ssh sessions if needed. Regards, Robot	[reply]
Re^2: SSH2 scp_get problem by RaduH (Scribe) on Oct 19, 2007 at 17:18 UTC
Well, for now I solved my problem using SCP. The System SCP, that is, not the Perl Net::SCP module. I just happened to have tried that first and it worked for me. I changed all my code to use that, even the part of Net::SSH2 that worked for me (scp_put works just fine). At least it is all nice and uniform now. I talked to the developer of the module and I files a bug report on CPAN, following his advice. Thanks, Radu	[reply]