Every time a new request comes in, the client transaction writes a JSON file into the queue directory. For the sake of no collisions, the file name can be a timestamp followed by some HTTP identifier -- perhaps the IP address.

The request script can then check if the agent is running by seeing if an agreed upon file is locked. If it is not, the request script forks off the agent. The advantage of a lock file here is that if the script exits abnormally, the lock is dropped, and so the agent will respawn on next request.

The agent moves through the queue, unlinking after each request is processed. Once the queue is empty, it exits.

If you do not require realtime response, you can also do this as a cron job. Fewer moving parts.

Finally, if you are worried about the DB fetches and they do not need to be current, you can use Storable or just use a local JSON file as cache.

Thx for the input. In my oppionion its goes down the same road if i query a memory table every second or list the content of a directory every second when i think it costs more to list the content of a directory over and over AND have to open the files to get the content. thats why i was looking for an event driven system.

You could convert either mechanism into an event driven system by using a named system semaphore,

Your super agent would create the named semaphore and then wait on it.

Your other code puts the information required by the super agent into a "known place"; whether that is a db table or the file system; and then signals (posts) the semaphore to tell the super agent it has work.

A perhaps simpler mechanism would be to have the super agent listen on a port. The other code delegates work to it by writing the details of the task to that port; and the tcp stack does the queuing.

The downside is that work items aren't persistent across system failures. The upside is work items aren't persistent across system failures.

---

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)

In the absence of evidence, opinion is indistinguishable from prejudice. Not understood.

You are missing the point of the spawn. There is a low utilization and high utilization case to consider.

Low utilization:
Requests are sufficiently infrequent that tasks complete before the next comes in. Agent is initiated for each new task and goes away when it is done, thus it only performs a query at the frequency of requests.

High utilization:
Requests come in faster than they can be completed. Agent is initiated by the first request of the cluster, and only once. For a file-based agent, main loop can look like:

while(my @list = `ls -t`) {
    chomp @list;
    for my $file (@list) {
        open my $fh, '<', $file or log_it_somehow;
        execute_json(<$fh>);
        close $fh;
        unlink $file;
    }
}
[download]

Note the directory content is queried an absolute minimum of times and the system handles sort orders. Also, note that the overhead in opening a file, reading it, and deleting it is highly unlikely to be greater than the overhead associated with database interaction.

Between the two cases, the agent is only running when there is need and there is no busy-wait. If you wanted to use a database rather than the file system, there's nothing fundamental to change to the algorithm. It's the idea of initiating on request that moves this into event driven space. You can certainly use a framework, but this is simple enough that a framework is probably overkill.

---

#11929 First ask yourself `How would I do this without a computer?' Then have the computer do it the same way.