Re: FTP Get Directory Tree Only

I have arrived at a solution, unfortunately a rather hacky one.

I used wget with a long long list of "rejected" files:

wget 
-R htm,txt,inc,ram,css,asx,jpg,gif,html,doc,xml,pdf,mp3,log,rtf,[etc] 
-r -nH --level=0 --force-directories 
ftp://<username>:<password>@<server>/<path>/
[download]

(the actual code was all on one line )

and then I used File::Find locally to go through and remove any files which accidentally made it through.

Of course any more interesting perl solutions would still be welcome. This could happen again...

($_='kkvvttuu bbooppuuiiffss qqffssmm iibbddllffss')
=~y~b-v~a-z~s; print

Comment on Re: FTP Get Directory Tree Only Select or Download Code

Replies are listed 'Best First'.
Re^2: FTP Get Directory Tree Only by roboticus (Chancellor) on Aug 15, 2006 at 12:24 UTC
Two other ideas: 1) I haven't tried it, but while reviewing the man page for wget, I noticed the option `--delete-after`. Perhaps it would leave the directory structure behind after getting and deleting the local files. 2) A while ago, i wrote a program using Net::FTP to get hierarchical collections of files. I just chopped it up to give you a starting point: #!/usr/bin/perl -w #============================================================== # GetDirHierarcy.pl #============================================================== use strict; use warnings; use Net::FTP; my $host = 'your.ftp.server'; my $uid = 'user_id'; my $pwd = 'pa$$word'; my $ftph = Net::FTP->new($host) or die "Can't connect to $host"; $ftph->login($uid,$pwd) or die "Can't login"; &GetFiles("."); $ftph->quit; sub GetFiles { my $pref = shift; # NOTE: Assumes UNIX ls format! #my @DirList = grep {/^d/} $ftph->ls; my @DirList = $ftph->dir; for my $DirEnt (@DirList) { next if $DirEnt !~ /^d/; my $DirName = (split / +/, $DirEnt, 9)[8]; my $newpref = $pref . '/' . $DirName; print "mkdir $newpref;\n"; $ftph->cwd($DirName); GetFiles($newpref); $ftph->cwd(".."); } } [download] Running this against an FTP server I have access to gives: `$ ./GetDirStruct.pl mkdir ./Files; mkdir ./Files/Bar; mkdir ./Files/Bar/Baz; mkdir ./Files/Foo; mkdir ./Inquiry; mkdir ./NpcIN; mkdir ./Processing; mkdir ./Security; mkdir ./Sysmenu;` [download] Just edit the print statement to actually make it create the directory hierarchy. NOTE: You may encounter an oddball FTP server that uses a different format for the DIR command. (I don't think that the results of the DIR command are in the FTP standard.) So you may have to edit the directory-name parsing portion of the code for your server. (I ran into this problem on an IBM mainframe FTP server.) --roboticus	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re^2: FTP Get Directory Tree Only
by roboticus (Chancellor) on Aug 15, 2006 at 12:24 UTC

--delete-after

Net::FTP

#!/usr/bin/perl -w
#==============================================================
# GetDirHierarcy.pl
#==============================================================
use strict;
use warnings;
use Net::FTP;

my $host = 'your.ftp.server';
my $uid = 'user_id';
my $pwd = 'pa$$word';

my $ftph = Net::FTP->new($host) or die "Can't connect to $host";
$ftph->login($uid,$pwd) or die "Can't login";


&GetFiles(".");
$ftph->quit;

sub GetFiles {
    my $pref = shift;
    # NOTE: Assumes UNIX ls format!
    #my @DirList = grep {/^d/} $ftph->ls;
    my @DirList = $ftph->dir;
    for my $DirEnt (@DirList) {
        next if $DirEnt !~ /^d/;
        my $DirName = (split / +/, $DirEnt, 9)[8];
        my $newpref = $pref . '/' . $DirName;
        print "mkdir $newpref;\n";
        $ftph->cwd($DirName);
        GetFiles($newpref);
        $ftph->cwd("..");
    }
}
[download]

$ ./GetDirStruct.pl
mkdir ./Files;
mkdir ./Files/Bar;
mkdir ./Files/Bar/Baz;
mkdir ./Files/Foo;
mkdir ./Inquiry;
mkdir ./NpcIN;
mkdir ./Processing;
mkdir ./Security;
mkdir ./Sysmenu;
[download]

NOTE:

[reply]
[d/l]
[select]