in reply to Re^2: Coro, AnyEvent and performance
in thread Coro, AnyEvent and performance

That all depends. If you don't need to share data at all, BrowserUk's solution may be simplest. If you need to share data, it becomes a bit more of a mess. Coro allows you to go on without worrying about simultaneous access to shared resources - you only need semaphoring for when you want to keep someone from modifying something while your thread may be ceded (your thread "blocks" on some asynchronous access). And this should be pretty rare. With full-blown threads, it can get more complex in a hurry - the more complex the data that you need to share, the more thought you need to provide. Not that it can't be worth it, but weighing the options is probably prudent.

Update: "more thought you need to provide" includes things like semaphoring, not merely the act of sharing a hash. (Though I do wonder if the copying of the non-shared hash to the shared hash is atomic - is there a race condition there?)

Replies are listed 'Best First'.
Re^4: Coro, AnyEvent and performance
by BrowserUk (Patriarch) on Jul 06, 2011 at 19:19 UTC
    the more complex the data that you need to share, the more thought you need to provide.

    That just is not true. You need to share a hash full of data, t'is easy:

    #! perl -slw use strict; use bignum; use threads; use threads::shared; use Data::Dump qw[ pp ]; my %results :shared; async { my %comps; for my $i ( 1 .. 1e6 ) { $comps{ $i } = $i ** $i; } %results = %comps; }->detach; while( sleep 1 ) { print "timer event"; last if %results; } pp %results;

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

      I don't think it's quite that easy. I updated the end of your code to look like this:

      my $c; while( sleep 1 ) { print "timer event"; $c = scalar keys %results; last if $c; } pp $c;
      Given that each entry in the loop should get its own key, I'm expecting 1e6 keys. That's not what I'm getting.
      $ perl5.14.1 x.pl timer event 15466 $ perl5.14.1 x.pl timer event timer event 436817 $ perl5.14.1 x.pl timer event timer event 445051 $ perl5.14.1 x.pl timer event timer event 499204
      If I change the sleep to 4 seconds, I get 1000000 consistently. This implies that more thought is required. A semaphore is required to ensure that the data required is present when going to read from it. For example, and this is probably not the best way in general (and definitely not in this case - knowing that it will be 1e6, it should be last if $c == 1e6, but, as a general rule, we may not always know how big the hash will be when it's done):
      #! perl -lw use strict; use bignum; use threads; use threads::shared; use Data::Dump qw[ pp ]; my %results :shared; async { my %comps; for my $i ( 1 .. 1e6 ) { $comps{ $i } = $i ** $i; } lock(%results); %results = %comps; }->detach; my $c; while( sleep 1 ) { print "timer event"; $c = do { lock(%results); scalar keys %results; }; last if $c; } pp $c;
      The downside is that some timer events are missed here while we're locked, waiting for the results to copy (one million keys and values can take a while). However, if we were to use a separate flag to signify done-ness, one that we know will get set when the hash is finished copying (so not another key in the same hash), we will catch all events and eliminate the race condition:
      my %results :shared; my $done :shared; async { my %comps; for my $i ( 1 .. 1e6 ) { $comps{ $i } = $i ** $i; } %results = %comps; $done++; # using $done = 1 doesn't want to work - more thought re +quired! }->detach; my $c; while( sleep 1 ) { print "timer event"; last if $done; } pp scalar keys %results;
      This will consistently produce 3-4 timer events (on my machine), and always output "1000000" when it's done. Like I said, more thought. I was originally going to write that complex data structures were hard-to-impossible, but I re-read the threads documentation before posting and realised that this wasn't necessarily true, so I revised it to "more thought" before posting. And I think I can safely stand by that statement. I've never done any threading in perl (only in C/C++), and I expect you've done much, yet you still missed some core issues with threading, not that this is perl-specific. You were probably in a hurry to dispel common misconceptions (which I admit that I had before re-reading the docs) and didn't take the time to fully think about it, inadvertently proving my statement: more thought.

      Update: I figured out why $done = 1 wouldn't work. It's the use bignum at the top. I think this is merely further evidence that threading in perl, especially when sharing variables, requires more thought. This would all work fairly trivially with Coro with the only thought being where to put the Coro::cede's (the original poster's question).

      A reply falls below the community's threshold of quality. You may see it by logging in.