Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

recursive SFTP

by xorl (Deacon)
on Apr 08, 2009 at 20:18 UTC ( [id://756449]=perlquestion: print w/replies, xml ) Need Help??

xorl has asked for the wisdom of the Perl Monks concerning the following question:

So I've decided my file transfer problem can be solved with SFTP (I'd prefer SCP, but you can see from my other posts that I've run into problems with it and I'll just say my last solution broke and I'm going to just give up on SCP for now).

While Net::SFTP looked like it would be great, it doesn't work since I need something that easily does a recursive upload.

I then found Net::SFTP::Recursive this sounded wonderful until I actually read the documentation. It appears you can only transfer files from the remote server to the local box. I need to go from local to remote.

Is there another SFTP module that can do a recursive upload easily?

I tried to write something my self using File::Find to find all the files on the local box, but I ran into trouble with Net::SFTP not creating subdirectories and it started getting really complicated with figuring out the correct directory anyway. If I have to write something myself, how do I get around these problem?

So to summarize I want to know:

  • 1) Is there a Perl module that will let me use SFTP to upload an entire directory structure (in other words a recursive upload)?
  • 2) If that's not possible, I need some help on writing something that will let me do that?

Thanks in advance.

Replies are listed 'Best First'.
Re: recursive SFTP
by shmem (Chancellor) on Apr 08, 2009 at 23:10 UTC

    An often used trick to transfer entire file trees is packing them. See CPAN. Lotsa *tar.gz files there.

    If that's not feasible, you'd need to create the directories on the remote site first, then transfer the files. If Net::SFTP::Recursive doesn't do that, you could still issue mkdir -p via ssh.

Re: recursive SFTP
by eye (Chaplain) on Apr 09, 2009 at 09:33 UTC
    If you happen to be working in a linux environment, you could approach this a bit differently by using the fuse kernel module with the lftpfs driver to let the OS manage the network connection (including the sftp protocol). Then the problem looks more like doing a recursive copy.

    Fuse has also been ported to FreeBSD and OS X, but I don't know if the lftpfs driver works on those platforms.

    Apropos of nothing, the lftpfs fuse driver is written in Perl.

    See:
    http://fuse.sourceforge.net/
    http://lftpfs.sourceforge.net/
    http://fuse4bsd.creo.hu/
    http://code.google.com/p/macfuse/

Re: recursive SFTP
by Bloodnok (Vicar) on Apr 09, 2009 at 11:01 UTC
    Could you not use cpio(1)/tar(1) & ssh e.g. something along the lines of ...
    find <dir> | cpio -omd | ssh <host> "cd <tgt dir> ; cpio -i"
    or similar ?

    For example, to transfer a local lib directory from a Linux host to an AIX host, I can do...

    pointo1d@unforgiven:~/workspace/IBM-Common-Collection$ find lib | cpio + -o | ssh root@aker "cd /tmp ; cpio -idm" 111 blocks 111 blocks
    Just a thought ...

    A user level that continues to overstate my experience :-))
Re: recursive SFTP
by salva (Canon) on Apr 09, 2009 at 14:13 UTC

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://756449]
Approved by GrandFather
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others admiring the Monastery: (3)
As of 2024-03-29 04:57 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found