You are missing the point of the spawn. There is a low utilization and high utilization case to consider.
Low utilization:
Requests are sufficiently infrequent that tasks complete before the next comes in. Agent is initiated for each new task and goes away when it is done, thus it only performs a query at the frequency of requests.
High utilization:
Requests come in faster than they can be completed. Agent is initiated by the first request of the cluster, and only once. For a file-based agent, main loop can look like:
while(my @list = `ls -t`) {
chomp @list;
for my $file (@list) {
open my $fh, '<', $file or log_it_somehow;
execute_json(<$fh>);
close $fh;
unlink $file;
}
}
Note the directory content is queried an absolute minimum of times and the system handles sort orders. Also, note that the overhead in opening a file, reading it, and deleting it is highly unlikely to be greater than the overhead associated with database interaction.
Between the two cases, the agent is only running when there is need and there is no busy-wait. If you wanted to use a database rather than the file system, there's nothing fundamental to change to the algorithm. It's the idea of initiating on request that moves this into event driven space. You can certainly use a framework, but this is simple enough that a framework is probably overkill.
#11929 First ask yourself `How would I do this without a computer?' Then have the computer do it the same way.
|