recursive SFTP

xorl has asked for the wisdom of the Perl Monks concerning the following question:

So I've decided my file transfer problem can be solved with SFTP (I'd prefer SCP, but you can see from my other posts that I've run into problems with it and I'll just say my last solution broke and I'm going to just give up on SCP for now).

While Net::SFTP looked like it would be great, it doesn't work since I need something that easily does a recursive upload.

I then found Net::SFTP::Recursive this sounded wonderful until I actually read the documentation. It appears you can only transfer files from the remote server to the local box. I need to go from local to remote.

Is there another SFTP module that can do a recursive upload easily?

I tried to write something my self using File::Find to find all the files on the local box, but I ran into trouble with Net::SFTP not creating subdirectories and it started getting really complicated with figuring out the correct directory anyway. If I have to write something myself, how do I get around these problem?

So to summarize I want to know:

1) Is there a Perl module that will let me use SFTP to upload an entire directory structure (in other words a recursive upload)?
2) If that's not possible, I need some help on writing something that will let me do that?

Thanks in advance.

Comment on recursive SFTP

Replies are listed 'Best First'.
Re: recursive SFTP by shmem (Chancellor) on Apr 08, 2009 at 23:10 UTC
An often used trick to transfer entire file trees is packing them. See CPAN. Lotsa *tar.gz files there. If that's not feasible, you'd need to create the directories on the remote site first, then transfer the files. If Net::SFTP::Recursive doesn't do that, you could still issue `mkdir -p` via ssh.	[reply] [d/l]
Re: recursive SFTP by eye (Chaplain) on Apr 09, 2009 at 09:33 UTC
If you happen to be working in a linux environment, you could approach this a bit differently by using the fuse kernel module with the lftpfs driver to let the OS manage the network connection (including the sftp protocol). Then the problem looks more like doing a recursive copy. Fuse has also been ported to FreeBSD and OS X, but I don't know if the lftpfs driver works on those platforms. Apropos of nothing, the lftpfs fuse driver is written in Perl. See: http://fuse.sourceforge.net/ http://lftpfs.sourceforge.net/ http://fuse4bsd.creo.hu/ http://code.google.com/p/macfuse/	[reply]
Re: recursive SFTP by Bloodnok (Vicar) on Apr 09, 2009 at 11:01 UTC
Could you not use `cpio(1)`/`tar(1)` & `ssh` e.g. something along the lines of ... `find <dir> \| cpio -omd \| ssh <host> "cd <tgt dir> ; cpio -i"` [download] or similar ? For example, to transfer a local lib directory from a Linux host to an AIX host, I can do... `pointo1d@unforgiven:~/workspace/IBM-Common-Collection$ find lib \| cpio + -o \| ssh root@aker "cd /tmp ; cpio -idm" 111 blocks 111 blocks` [download] Just a thought ... A user level that continues to overstate my experience :-))	[reply] [d/l] [select]
Re: recursive SFTP by salva (Canon) on Apr 09, 2009 at 14:13 UTC
Use Net::SFTP::Foreign!	[reply]