I'd be doing this via rsync, but my webhost doesn't have sshd or rsyncd running.
I'm guessing you don't have login shell access on the remote server, since this would most likely mean that sshd would be available.
Maybe you've got the wrong webhost, unless they can tell you specifically what sort of secure login / file-transfer protocol they support -- i.e. one that is not anonymous and does not involve the transmission of login transactions (username / password) as clear text over the public network. Maybe there's a way that this is done without using sshd? (I don't know -- everything I've come to be familiar with, ever since login sniffers became a known problem, has used sshd one way or another.)
It's not so much a matter of protecting your data; it's a matter of protecting access to the storage -- i.e. the service provider should be at least as worried about this as any customer would be. If they don't care about it, they might not be worth whatever you may be paying for their service. (If the service is free, it might just be worth what you're paying for it.)
That said, I'm wondering how closely you've looked at the problem you've described. In terms of using file dates as listed by the remote ftp server (and doing time-zone conversions on that), bear in mind that, depending on the remote OS, the Net::FTP "dir" method might return different strings for the same file over time.
Unix(-like) machines will often show different date information (year vs. hour:minute) depending on whether the file is more than 6 months old. So today (March 11 2007, say) Net::FTP might show a file date as "Oct 24 14:45" (meaning Oct. 2006 -- but if you just pass that quoted string to a module like Date::Manip, it'll treat it as Oct. 2007). A month or so from now, it will be shown as "Oct 24 2006".
On top of that, ftp transfers cannot force a copied file to assume the modification date of the original -- when you upload a file to the server, the file date on that copy reflects the time of transfer, not the modification time of the file on your local machine. This, combined with what appears to be lax security on the part of the service provider, suggests that when the remote file is more recent than the local, this might not be sufficient evidence that the remote version is identical to your local file. (Having the same size might not be sufficient proof, either.)
So maybe you need to maintain a list of paths, names, dates and checksums, for both local and remote data sets. I would start by using the standard unix "find" utility to scan the local tree, list all the data files, and produce md5 checksums for them -- best done with a single command line:
(update: fixed error involving the return value of -M)find top_level -type f -print0 | xargs -0 md5sum | perl -lape 's/ /$d=(-M $F[1])*3600*24;sprintf(" %9d %s ",-s _,sc +alar(localtime($^T-$d)))/e' > master.list
(If you're local box is ms-windows and you haven't got unix tools installed yet, just download cygwin or uwin to get "find" and "xargs".)
Note that the perl one-liner there is adding date and file size information, and doing the date string that way will always produce a consistent date format. (You could elaborate that a little, adding Date::Calc or some such to output dates that reflect the timezone of the remote server.)
Next you would need a perl script to load that list and the list of "known exceptions", and then uses Net::FTP to scan the the remote directory tree recursively. Assuming the lists are hashes keyed by path/file.name, you can easily identify files on the server that shouldn't be there (not found in the hash lists), and local files that don't exist on the master (in the hash list, but not on the remote machine).
For files with versions on both machines, you have more work to do. Using the ftp "dir" output to compare dates and sizes might suffice, especially if you've kept track of dates and times when you uploaded each of the data files (so you can see if any file date changes after you upload it). But I won't go into that, because this reply is already too long, and you really should just work on getting access to rsync or sshd on the server, even if it means switching providers.
In reply to Re: Sync via FTP
by graff
in thread Sync via FTP
by Tanktalus
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |