in reply to Re^2: Doing an Input->process->Output cycle with AnyEvent? (was: error: "recursive blocking wait detected")
in thread Doing an Input->process->Output cycle with AnyEvent? (was: error: "recursive blocking wait detected")

So when I needed to add this input/output interface here, a component which influences the various parts of the system, I thought it would be a consistent move to model this with AnyEvent's tools as well. Wrong asumption?

Hm. Let me be up-front and say: I don't understand why anyone would voluntarily choose to use an event-driven system on a pre-emptive, multi-tasking operating system. The phrase that comes to mind is: "Like buying a dog and barking yourself".

It always reminds me of one of those plate-spinning acts. The guy or girl is running around splitting their time between trying to load up new plates onto poles, whilst keeping the existing ones spinning fast enough to stop them falling. Everyone gets behind them cheering them on. Unwittingly helping them out by collectively "whoooa"ing, whenever one of them starts to wobble violently. Eventually they either complete the poles available or their timeslot comes to an end and everyone applauds with admiration. But in the end, we all know they're all going to come crashing down.

In the past, I've had to program such systems, because there was no other option available. They can be made to work well, and can be very efficient. But at what expense? They are so complicated and fragile. Difficult to program; a nightmare to debug; and horrendously expensive to maintain. One change to the specification; or the need to move to different hardware; and you're into a game of having to re-tune every part of the system.

And once you're locked into a given framework, nearly everything you want to do requires a "special version" of everyday modules. Just witness the hundreds of modules that turn up when you search for AnyEvent::* or POE::*. And see how many of them are specially adapted versions of common, well used modules like ...::LWP::* and ...::NET::* and ...::IO::*. And then there are the dependency chains. A bug arises and a fix is a applied to one of those core modules, but then you have to wait for that fix to make it's way through the chain into the specialist version you're using.

Their proponents will tell you horror stories of deadlocks and inversion chains, and all sorts of other nasties, if you bring up threads; but as you've discovered, you don't avoid synchronisation issues with an event-based system. Indeed, you have to use it to try re-create a simple serial sequence of DoA() then DoB() then DoC().

You want to read-process-write, see the loop above. You need to do other things while that goes on:

async{
    while( <> ) {
        ## process $_
        print;
    }
};

[download]

Now you can!

Let that well-tested, highly tuned, pre-emptive multi-tasking scheduler take care of making sure that when things need doing, they get done. None of this interrupting everything you're doing, every 1/10th of a second, to run around everywhere else asking (polling): Does anything need doing? Does anything need doing?

Don't feel the need to counter this--it's just my view on things--but I've yet to see an event-driven application that wasn't simpler written and maintained using threading. That's actually a challenge that I'd love to see someone take up. But I doubt they will. I'll let you decide the reason why not :)

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
RIP an inspiration; A true Folk's Guy
  • Comment on Re^3: Doing an Input->process->Output cycle with AnyEvent? (was: error: "recursive blocking wait detected")
  • Download Code

Replies are listed 'Best First'.
Re^4: Doing an Input->process->Output cycle with AnyEvent? (was: error: "recursive blocking wait detected")
by isync (Hermit) on Oct 17, 2010 at 01:12 UTC
    Thanks for your opinionated post! So far, I've only taken a few steps into the event-framework-world. After reading a bit of POE docs I soon tripped about more and more comments which said "avoid POE". AE promised to master them all. And when I started to dive into AE, exactly what you've described began to happen: I found myself writing a while loop again AND event triggers and listeners etc.

    An old-fashioned serial loop structure plus some threading might not have the bells and whistles of events, but your post convinced me to try the "pragmatic path" first. Keep it simple stupid, once again. I think I need to re-think my design...
[reply]
      Yes, from your description you seem to be trying to shoehorn a sequential program into an event loop. AnyEvent (or POE, or IO::Async, or whatever) is not a hammer for every nail, it is not always appropriate, but I'm not certain I'd recommend threads either, perhaps a combination of fork and some form of IPC (using one of the fine modules on CPAN, such as Parallel::ForkManager) might be better. It would definitely make good use of your pre-emptive, multi-tasking operating system.
[reply]
      I think I need to re-think my design...

      If you are able to post a description of the application, I'd be more than willing to offer suggestions on how you might tackle it with threads.

      Maybe we could even get one of the events guys to knock up a solution also and then compare results.

      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
        I think I will now revert to doing more actual development on the system after all the input, to find out where the real challenges lie.
        ...before I begin to post gory details which might turn out as being sideshows... ;)

        Thanks everyone!
[reply]
Re^4: Doing an Input->process->Output cycle with AnyEvent? (was: error: "recursive blocking wait detected")
by andal (Hermit) on Oct 20, 2010 at 08:52 UTC
    Hm. Let me be up-front and say: I don't understand why anyone would voluntarily choose to use an event-driven system on a pre-emptive, multi-tasking operating system. The phrase that comes to mind is: "Like buying a dog and barking yourself".

    Really. I don't understand, why anyone would voluntarily choose to use multi-threading in situations when there are lots of interactions between threads. The mess of mutexes and semaphores with really high chance of dead-locking - that's what such application looks like.

    The rule is simple. Use tools that are appropriate for the work. If the actions are short and interaction is high, then event-driven system is the best. If the actions take longer time and don't interact much, then multi-threading is your choice. And of course, don't bother with all of this if your application is very simple. This approach will speed up the execution and simplify the development. All you need is good understanding of the event-driven approach.

[reply]
      The mess of mutexes and semaphores with really high chance of dead-locking - that's what such application looks like.

      Really? Show me. Or better yet. Show me, (real working code I can download), that does something (other than GUI or webserver), for which you believe that an event-driven architecture is the right tool for the job.

      In return, I'll try to reproduce that using threads, without the need for a "mess of mutexes and semaphores".

      All you need is good understanding of the event-driven approach.

      Please believe me when I tell you that I do understand event-driven architectures.

      • From 6502-based assembler controlling z80-based OMR readers;
      • to IBM PCs running DOS talking 3270 protocol to System/38s via 3274 controllers;
      • to doing concurrent, bulk data transfers to and from up to 64 hand-held terminals via serial ports from C code under DOS.

      It is exactly because I've done it, and then had to go through the pain of re-tuning it (the last project above), to run on faster hardware. And then add more ports and re-tune again. And then add a remote monitoring and control channel via modem, and the re-tune the whole, internecine monster all over again.

      It is exactly because of that, that I don't understand why any one would choose the architecture if they had a choice.

      It worked, and worked well (most of the time). But sheesh! Was it ever a bitch to maintain, upgrade and debug.

      Latterly on the above project I ported it to OS/2 and used threading. What a joy. Simple, linear flows: read-ack-read-ack-read-nak-write ack. Add more ports, start more threads. Waiting for IO, just block. Need an accurate time window, raise your priority and sleep for the required number of milliseconds. Someone dials in, start a thread to talk to them. Each thread has it's own context and its own stack. No constantly shuffling data in and out of global stores each time you have to deal with something else.

      Really, I've done it both ways, and I know which is easier to write, maintain and debug. Or at least, I believe I do. So prove me wrong?

      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

        I would point you towards the "other" event oriented protocols, like irc clients. They and the protocol lend themselves well towards event oriented approaches. But they are quite similar to webservers in their nature anyway.

        Maybe all systems where events come in and get dispatched are such a class? Although I see the advantage if you have multiple clients and more complex interactions than request -> response, where the linear flow in programming is preferrable over the event oriented approach, where you stitch together the flow by keeping some additional state.

[reply]
[d/l]

In Section Seekers of Perl Wisdom