in reply to What the flock!? Concurrency issues in file writing.

I just have to ask, because it hasn't been mentioned... You are not writing to an NFS filesystem, right?

You could use a semaphore instead of flock (IPC::SysV::Semaphore).

Of course, writing lots of data from multiple sources for ultimate sorting sounds like a DB application to me. SQLite or MySQL might seem to be overkill, but they are really easy to install and interface with. If you are storing these reports over time, this is a much better archive system as well. Finally, if your retrieval needs increase in the future, your options are wide open.
  • Comment on Re: What the flock!? Concurrency issues in file writing.

Replies are listed 'Best First'.
Re^2: What the flock!? Concurrency issues in file writing.
by suaveant (Parson) on Oct 01, 2008 at 18:50 UTC
    Semaphore doesn't work, either... it must be something at the file buffer level or something, but I've tested syswrite and it returns the proper number of bytes written.

                    - Ant
                    - Some of my best work - (1 2 3)

[reply]
Re^2: What the flock!? Concurrency issues in file writing.
by suaveant (Parson) on Oct 01, 2008 at 18:23 UTC
    Not NFS.

    I am only sorting to get the data back in line after parallel writing, since data gets written out of order (and is not fixed, so I can't just seek to the proper position and write)

                    - Ant
                    - Some of my best work - (1 2 3)

[reply]

      If you have to sort the results anyway, just dodge the bullet and have each of your processes write to separate files. Then sort the files into one. You can't sort until you get the last line anyway, so there is no loss of parallelism.

      Also, a threaded solution is possible if you are interested.

      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
        I've thought about both of these, and I may actually look into the multiple file option since I could possibly write these subfiles in order, if I tweaked everything right.

        Threading is definately something I have thought of but would be a big change, not to mention getting up to speed on threads :)

                        - Ant
                        - Some of my best work - (1 2 3)

[reply]

In Section Seekers of Perl Wisdom