You could keep track of a time stamp on the last file fetch date and retrieve only the files whose last modification date is more recent.

Of course, if the directory is very dynamic, you might get files that are still being written (depending on your OS: for example, it wouldn't be possible to read a file that is being written under VMS, but that's entirely possible under most Unix/Linux flavors). That's not the question you asked, but it is often possible to find a delay that will prevent working on files being written.