Re: Monitoring directory for new files

You will be notified that something has changed in the "drop directory". When you see that happened, you read the directory and process all files that you know of.

If a file isn't finished being created, your read loop will hang while waiting for EOF. That appears to be fine in your app.

The potential race condition comes into play after you have finished processing all files that you knew about when you got the triggering event.

It could be that a new file has been or is being created in the "drop directory" since you last checked. So you run the "process_all_files()" routine again to make sure that the directory is empty as far as you know.

But this is may still not be perfect. Unless you have good OS event support and I'm not sure that Windows does, you should run process_all_files() when the event trigger happens, then run process_all_files() again to clear the directory "to the best of your knowledge". And then run process_all_files() periodically so that no "files get stuck" in the drop directory without an event trigger.

There are various alternate schemes, but the above is a good one provided that you can deal with a little delay when a file "gets stuck", which won't happen often.

Comment on Re: Monitoring directory for new files

Replies are listed 'Best First'.
Re^2: Monitoring directory for new files by ikegami (Patriarch) on Jan 20, 2010 at 06:01 UTC
If a file isn't finished being created, your read loop will hang while waiting for EOF. I'm sure this is wrong (since we're talking about plain files, not pipes or sockets), but I can't test it right now.	[reply]
Re^3: Monitoring directory for new files by Marshall (Canon) on Jan 20, 2010 at 06:12 UTC
If that's not true, then what happens? I figure the reading process will keep trying to read until EOF. It will not see EOF until the other process which is writing the file has closed it. I could be mistaken, but again if it doesn't work like this, then what happens in the "reading process"? ~~So basically, yes this is like a pipe.~~ I think Ikegami's test is correct.	[reply]
Re^4: Monitoring directory for new files by ikegami (Patriarch) on Jan 20, 2010 at 15:53 UTC
You are mistaken. Tested on Win and Linux. It stops reading when it reaches the current EOF, saying that it reached EOF. It doesn't matter that some process still has an open file handle to the file. If it was a pipe or socket, it would block as you describe.	[reply]