in reply to efficient ftp of numerous files

You want to fetch recently changed files, to update a local repository.

I believe rsync is suitable for the task.

--
TTTATCGGTCGTTATATAGATGTTTGCA

Replies are listed 'Best First'.
Re: Re: efficient ftp of numerous files
by biosysadmin (Deacon) on Mar 09, 2004 at 16:06 UTC
    Agreed. While you could write something in Perl to solve this problem, there's no point in re-inventing the wheel. if you don't have it on your system it's available from rsync.samba.org. On the other hand, it sounds like you may not be creating a mirror of the entire repository of pictures, just making a local mirror of a picture set whose members change nightly. If the latter is the case, there's a few fun ways to approach this:
    • If you have shell access to your remote site, perhaps you could write a script that created a directory of symbolic links to the files you want to download. You could then run the script nightly and then use rsync to create a local mirror of the remote directory.
    • If you don't have shell access to the remote site, then you could make the hash like you describe above.
    Good luck with it. :)
[reply]

In Section Seekers of Perl Wisdom