mcrose has asked for the wisdom of the Perl Monks concerning the following question:

Given the following code, the runtime and performance depends heavily on where I actually put my poll for new events. If I poll every time through the loop, I get a horrendous performance hit. I can only poll every once in a while by modding against the current iteration, but I'd like to use this in situations where I may not be maintaining an iteration counter or when one doesn't make sense.

Is there a way to simply integrate events into Coro by telling the threading mechanism to poll for new events every n ms or so?

use strict;
use warnings;
use 5.012;

use Coro;
use AnyEvent;
use Coro::AnyEvent;

my $w; $w = AE::timer 0, 1, sub { async { say "timer event" } };

for my $i (1 .. 1000000) {
  my $computation = $i ** $i;
  if ($i % 10000 == 0) {
    say $i;
    # Coro::AnyEvent::poll; # fast - ~200k iterations/sec 
  }
  Coro::AnyEvent::poll; # slow - ~25k iterations/sec
  cede;
}

[download]

Replies are listed 'Best First'.
Re: Coro, AnyEvent and performance
by BrowserUk (Patriarch) on Jul 06, 2011 at 18:35 UTC

    Instead of mucking around with Windows 3.0-esque pseudo-threading and polling, you could just use proper threading:

    #! perl -slw
use strict;
use bignum;
use threads;
use threads::shared;

my $result :shared;
async {
    my $comp = 0;
    for my $i ( 1 .. 1e6 ) {
        $comp += $i ** $i;
    }
    $result = '' . $comp;
}->detach;

while( sleep 1 ) {
    print "timer event";
    last if $result;
}

print $result;

    [download]

    It's simpler, and actually works!

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]
Re: Coro, AnyEvent and performance
by Tanktalus (Canon) on Jul 06, 2011 at 17:03 UTC

    What you want is for a kernel running outside of your application to forcefully take the CPU from you whenever it feels it necessary to do so. That is True Threading, which Coro is decidedly not.

    Your problem in your example code is that your loop is very, very short. You're computing a simple exponential (ok, near the end it's not so simple). The time for Coro/AnyEvent to figure out if there are any other events to process, process them, and come back to you, quite simply overwhelms your loop.

    If your loop was doing "real work" in the middle, you might find the overhead of poll/cede to be far less significant.

    For example, if your loop was using, say, AnyEvent::HTTP to query a web site, you wouldn't even need to poll/cede. Or if your loop was rendering some Template Toolkit output, that may be heavy enough that ceding each time through the loop may be minor.

    The reality is, these things have overhead that they need to go check everything before coming back to where you are. As long as you can push your hard work off into asynchronous pieces of work that need to wait for things (such as sockets, e.g., TCP/IP requests, or user events), Coro will net you a speed up. But if you're doing something computationally trivial between cede's, that's not going to seem like a benefit :-)

[reply]

      That's what I suspected was the case. So in a situation where I want to continually interleave a somewhat trivial computational task as a background thread but still stay responsive to intermittent events (say an order or two of magnitude less actual computational work), I'd be better off just forking off the computational task, having the parent run the event system, then passing back the computational results as an event to the parent via IPC?

[reply]

        That all depends. If you don't need to share data at all, BrowserUk's solution may be simplest. If you need to share data, it becomes a bit more of a mess. Coro allows you to go on without worrying about simultaneous access to shared resources - you only need semaphoring for when you want to keep someone from modifying something while your thread may be ceded (your thread "blocks" on some asynchronous access). And this should be pretty rare. With full-blown threads, it can get more complex in a hurry - the more complex the data that you need to share, the more thought you need to provide. Not that it can't be worth it, but weighing the options is probably prudent.

        Update: "more thought you need to provide" includes things like semaphoring, not merely the act of sharing a hash. (Though I do wonder if the copying of the non-shared hash to the shared hash is atomic - is there a race condition there?)

[reply]

Back to Seekers of Perl Wisdom