We have a mainframe process that drops off files into a shared directory. I have four Perl processes that attempt to read from the directory and process those files.

I'm looking for a way to synchronize the four Perl processes to keep them from each trying to process the same file.

My original plan was to File::Copy::move() the files to a local /tmp folder, and only the first guy to move the file would win (if your move failed, you just move on to the next file).

Turns out this fails because File::Copy::move() is actually implemented as a multi-step copy/set-attribs/delete process that doesn't have any file locking protecting it. I ended up with multiple processes each copying the same file, one would delete the source file, and the other would be just getting done and try to stat() the original file (to set the atime and mtime of the new copy) and there'd be no original file.

How can I keep these four processes (each on different servers, incidentally) from clashing and trying to process the same file?

-- Mitch

In reply to Synchronizing multiple processes retrieving files out of shared directory by MitchInOmaha

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.