in reply to Directory comparisons
If any files exist in both local directories, then I don't want the script to retrieve these files from the remote directory
This statement is not easy to understand, and it kinda contradicts to what you're saying before it and after.
Anyway, why don't you use the same List::Compare to compare local directory to a third (also local) directory? Some sample data to demonstrate what you're trying to achieve would be welcomed.
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re^2: Directory comparisons
by joeymac (Acolyte) on Dec 20, 2011 at 14:28 UTC | |
Here is (most of) the script. At least the relevant bits! In this version I have gotten the intersection of the two lists, but what to do with it?
| [reply] [d/l] |
|
Re^2: Directory comparisons
by joeymac (Acolyte) on Dec 20, 2011 at 14:10 UTC | |
What happened was there was a connection failure with the remote server and when the script was restarted, some duplication of data occurred and it has sporadically occurred since. Oddly enough, it never happened in operation before the remote server failure. My script retrieves from the remote directory to the local directory, then the files are picked up by other scripts out of my control and placed into the third (also local) directory (among other places). From there they are distributed to customers/websites (scripts also out of my control), so duplication of data is a big no-no. I am considering trying to use the $lc->get_intersection command (List::Compare) to find files that appear in both directory lists, but I'm not sure the best way to check that files here have not already been transferred. Someone has suggested a rotating stand alone file that serves as a "memory" of which files were successfully transferred during a given time frame, but this sounds like a bit of work. | [reply] |
by zwon (Abbot) on Dec 20, 2011 at 15:22 UTC | |
but I'm not sure the best way to check that files here have not already been transferred And by transferred here you mean that file was retrieved to local directory, copied to the third directory, and then removed by some other process, is it? In this case yes, you need to keep log of files which were successfully transferred (or rather just list of files that were copied to the third directory). | [reply] |
by joeymac (Acolyte) on Dec 20, 2011 at 15:40 UTC | |
Yes, sorry for the vagueness. That is what I mean by transferred. So the log file(s) should be something like a "dump" of the array contents (directory listing). Then read back in this log file, maybe splitting contents by filename and comparing to the directory list of things to be retrieved. And if it is in the log file, don't transfer it (again). Sound right? | [reply] |