Otherwise, I'd use a lockfile, or a locking mechanism in a database. You say flock() doesn't work, but I fail to understand why. Since only one process at a time can have an exclusive lock at one time, this seems like an ideal mechanism.

Abigail

Actually, they DO have to run in parallel, or else the results will take several minutes to generate, rather than just 10-15 seconds. This is crucial, as no one wants to wait long for a multi-headed search engine to print out results.

As for flock() not working, it's not that it doesn't work, it's that I had the same logic issue as with a flag. But I think I've found the way around that. I'm going to write it up, test it, then post it back here to this thread once I've got it worked out.

As for flock() not working, it's not that it doesn't work, it's that I had the same logic issue as with a flag.

I don't understand your logic issues. That is, if you want only one process to act at once (as you said in your original post). But now that you say that you do want them to run in parallel, I'm confused. What do you really want? One-by-one, or in parallel? You can't have both.

Abigail

That's why flock is nice for that, if you have that. I have an example of that in my famous HIGHLANDER article.

I've worked this out to something of the following design:

if (locked) {
     while (locked) {
          if (unlocked) {
               lock
               print
               unlock
               break;
          }
     }
} else {
     lock
     print
     unlock
}
[download]

The only thing I need to make this work is a flock() test to see if it CAN to an exclusive lock (the lock file is not already locked).

          if (unlocked) {
               lock
[download]

No, you've got a race condition there. Two processes can both see "unlocked" at the same time, then they both "lock" and they think they've got it.

That's why you need "atomic test and set". There's really no way around it. The thing that checks for unsettedness needs to also set it if it can, in a way that can't be interrrupted or observed by another process/thread/entity. This usually requires operating-system-level locks.

-- Randal L. Schwartz, Perl hacker
Be sure to read my standard disclaimer if this is a reply.

To make merlyn's point clearer, what you need is

sleep(rand) until lock(exclusive);
print
close
[download]

The lock() function must return whether it was able to get a lock, and you must not use a separate is_locked() function in this process. Actually, you should never need one at all, because you know when you have the lock and you're not interested in whether someone else has one if you don't have it.

sleep() is only necessary if your locking function does not block. rand() must be used if this is only one of several locks or other synchronization mechanisms used by multiple processes, because they might otherwise get in a deadlock situation when each process holds a lock on some of the resources but not on others.

You may also unlock() instead of close() if you know what you're doing, but it has only become safe in recent versions of Perl, and even so it poses the danger that you'll attempt to read data from the file while another process is writing to it. Ideally, you should lock(shared) if you're going to be reading from the file, and the locking mechanism should honour exclusive lock requests from others before your lock degradation request. In that way, no writing process starves, while no reading process gets to read inconsistent data.

Do not underestimate the complexity of locking. Unfortunately if you don't get it perfect, you might as well not bother at all.

Makeshifts last the longest.

my $lock = 1;

$SIG{CONT} = sub { $lock = 0 };

while($lock) { sleep 1 }

# Rest of your program
[download]

Then you send a CONT (continue) signal to the process when you want it to go. Probably not the best solution, as signal catching is considered a last-resort situation.

Part of the point of separate processes is that they cannot access each other's memory. So no matter how you pass \$lock to a different process, it won't be able to do anything useful with it.

You need to use flock (or something similar) but on a different file for each user and this filename is what you pass to each subprocess, not \$lock. I'd use File::Temp to create these lock files safely.

Update: The original node says:

flock() is not an option for this, as this program will need to be able to be run by multiple users at the same time.

Or, if you need only one lock, open $0 and flock it. Unique to the script!

-- Randal L. Schwartz, Perl hacker
Be sure to read my standard disclaimer if this is a reply.

$0 may be unique to the script, but that is a BIG problem. What I have is a main program (parent) spawning a number of child programs. Each of the child programs is trying to print to <STDOUT>. What I am having is when 2 children are trying to print to <STDOUT> at the same time, the information gets mixed. By trying to set flock(), I am trying to prevent one child from printing to <STDOUT> while another child is in the middle of doing that. The end result should be that I don't have any mixed outputs.

Problem is, the traditional uses of flock() aren't helping, they are creating either the same problem with the output, or they are creating runaway processes.

I've created a file, "lock_file". I have a lock_ex subroutine that returns a file handle when it can create an exclusive lock. I need to check to see if that is in place, if so wait, if not go ahead and print to <STDOUT>.

If I were doing this on Win32, I'd look at using Win32::Semaphore.

There is IPC::Semaphore which does a similar thing under *nix, but I don't have any experience of it, so I don't know if it could be adapted for your purposes if that is your platform.