in reply to Synchronizing multiple processes retrieving files out of shared directory

The "single point of failure" is the entire application inclusive of all of its worker-bees, and the "failure" that can occur is that any or all of those "bees" misbehave. Therefore, focus simply on the strongest possible design. In this case, have one process or thread whose only purpose is to recognize that a new file has arrived. Let that process/thread add an entry to a "work-to-do" queue, which is read by all of the "bees." Now, those "bees" never have to worry about interference. Let all of the processes, including both the bees and the directory-watcher, be owned by a parent-process whose only purpose in life is to make sure that none of its children die.
  • Comment on Re: Synchronizing multiple processes retrieving files out of shared directory

Replies are listed 'Best First'.
Re^2: Synchronizing multiple processes retrieving files out of shared directory
by MitchInOmaha (Initiate) on Mar 28, 2014 at 15:31 UTC
    That doesn't seem as easily done when those processes live on four separate servers. Although, I'm all for some IPC that could manage those processes in a Linux realm.