'background' CPU process when Idle in AnyEvent

sectokia has asked for the wisdom of the Perl Monks concerning the following question:

Consider the code below, where CPU crunching is done when Event loop is idle, but only for 0.1s to allow HTTP requests about the processed data to not be delayed more than ~100mS.

Is there any better way to do this that would eliminate the (up-to) 100mS wait time for a request to actioned? Basically... is there any way idle could run continously and be interuptted by the events? What is generally used in this situation. A second thread?

Thanks!

use strict;
use warnings;
use AnyEvent;
use AnyEvent::HTTPD;
use Time::HiRes qw(gettimeofday);
use Digest::MD5 qw(md5);

my $httpd = AnyEvent::HTTPD->new(port=>9090);
my $count = 0;
my $foo = 'bar';

# Handle other events
$httpd->reg_cb(
    '/count' => sub {
        my ($h,$r)= @_;
        $r->respond({content=>['text/plain',"$count: ".(unpack 'q',$fo
+o)]});
        $h->stop_request;        
    });

# CPU crunching for 100mS when idle...
my $idle = AnyEvent->idle(cb => sub {
    
    my $now = gettimeofday;
    while(gettimeofday < ($now + 0.1)) { $foo = md5($foo); }
    $count+=1;
});

my $cv = AnyEvent->condvar;
$cv->recv;
[download]

Comment on 'background' CPU process when Idle in AnyEvent Download Code

Replies are listed 'Best First'.
Re: 'background' CPU process when Idle in AnyEvent by haukex (Archbishop) on Jun 08, 2022 at 11:16 UTC
CPU crunching is done when Event loop is idle, but only for 0.1s to allow HTTP requests about the processed data to not be delayed more than ~100mS. Time slicing is a way for OSes to implement "parallel" processing on (single-core) CPUs, so it seems like you've reinvented that :-) So: What is generally used in this situation. A second thread? Yes, or second process, or some other way of running multiple processes in parallel; there are various ways to implement communication between them. Though I don't know much about AnyEvent, a quick search reveals e.g. AnyEvent::Fork, whose documentation also links to several other similar modules in the AnyEvent family. Update: Turns out I showed you an example of this using Mojo::IOLoop::Subprocess almost exactly two years ago.	[reply]
Re: 'background' CPU process when Idle in AnyEvent by NERDVANA (Priest) on Jun 09, 2022 at 00:29 UTC
The "recommended way" depends a lot on what your application is actually trying to accomplish. Event-driven HTTP servers work best when the web requests are quick or mostly waiting on other servers (such as a database) Forking HTTP servers work best when the code that serves requests uses blocking APIs, or when the request needs to do a lot of processing. Perl has a single-threaded design, so if you have two distinct tasks you want it to do like 1) respond to external events (like HTTP requests) and 2) run a big complex calculation, the best way to do it is to have two completely different programs doing these things. One program could spawn the other and manage it as a process parent, or you can run them both from some sort of service manager like Systemd or Docker or Runit. If the most important task is the big calculation, and you want to be able to check on its status using HTTP, then consider just generating a detailed log file from the calculation and let the HTTP server report info parsed from the log. If the most important task is responding to HTTP, and sometimes the http requests need to kick off a long-running background job, consider a task server like Minion.	[reply]