GaijinPunch has asked for the wisdom of the Perl Monks concerning the following question:

Hey monks. I've got a script that uses Net::FTP. For all intents and purposes, it works well. However, I'd like to do "mget *" where I can download a whole directory. Apparently Net::FTP doesn't support this. Any suggestions?

Cheers UPDATE: I used LWP to grab the URL that has the file list. Ran a regex on each line, made a file list like that, and will now feed it to the $ftp->get() function.

20040915 Edit by castaway: Changed title from 'Mget?'

Replies are listed 'Best First'.
Re: "Net::FTP" missing "mget"
by mifflin (Curate) on Sep 14, 2004 at 03:41 UTC
    You want to get all the files in the current remote directory right?
    Use the ls() method to get a list of all the files and then do individual get() calls for each file in the list.
      Yeah, I thought about using ls, but apparently the files are hidden? Not sure why. If I do $ftp->get(/path/to/file) it gets it. But cwd'ing to the directory, and doing ls, returns a null list. :(
        A few suggestions...
        • try dir
        • try list
        • try nlst

        FYI: the FTP protocol has no "mget" concept, clients typically impliment it using the NLST command...

        NAME LIST (NLST)
        This command causes a directory listing to be sent from server to user site. The pathname should specify a directory or other system-specific file group descriptor; a null argument implies the current directory. The server will return a stream of names of files and no other information. The data will be transferred in ASCII or EBCDIC type over the data connection as valid pathname strings separated by <CRLF> or <NL>. (Again the user must ensure that the TYPE is correct.) This command is intended to return information that can be used by a program to further process the files automatically. For example, in the implementation of a "multiple get" function.
        If I do $ftp->get(/path/to/file) it gets it. But cwd'ing to the directory, and doing ls, returns a null list.

        I think this has nothing to do with Net::FTP or any sort of mget or ls function. It has to do with how permissions are set on the particular directory on the ftp server.

        There's a little trick that the ftp maintainer can play with directory permissions, to provide a certain amount of "access control" or "data security" for files in the given directory. On a unix system, you can do this to a directory:

        chmod 751 some_directory
        which causes it to have permission (mode) flags like this: "rwxr-x--x", which means:
        • first "rwx": the owner can read (i.e. search), write (i.e. create/rename/delete files), and "execute" (i.e. access files) within the directory
        • second "r-x": the group associated with the directory can read (search) and execute (access files)
        • third "--x": everybody else, usually including the anonymous ftp user, can only access files -- but since they cannot search, they must know the exact name of each file they want to access.

        So, this sort of permission allows the ftp server to provide some data protection -- that is, to limit distribution to specific recipients -- because only the owner can put files in, and the only people that can get files are those in the owner's group, or else people who have been told exactly what file names to retrieve.

        This is also commonly used on ftp "upload" directories, where anon.ftp users can "put" files -- here the permission would be "rwxrwx-wx": anybody can create files in the directory, but only owner and group can search for file names. If one anon.ftp user puts a file into such a directory, other anon.ftp users won't know it's there, unless they are told what the file name is by someone who knows.

        (updated to fix grammar)

Re: "Net::FTP" missing "mget"
by jbodoni (Monk) on Sep 14, 2004 at 03:43 UTC
    Probably not what you're hoping for, I admit, but I've had a great deal of success by using Perl to generate a script file that is in turn fed to the ftp executable.

    John

Re: "Net::FTP" missing "mget"
by McMahon (Chaplain) on Sep 14, 2004 at 14:34 UTC
    I've done it like this:

    my @files = $ftp->ls(); foreach my $file(@files) { $ftp->get($file) or die "get failed", $ftp->message; }