http://qs1969.pair.com?node_id=865829

Originally written in response to:

..before I begin to post gory details which might turn out as being sideshows..

Though the nice thing about sideshows is that you don't have to look. You can carry on to the main event as if they did not exist. But, for some, the sideshows can be more interesting than the main event. If there was no interest, they wouldn't exist.

For the OPs of most SoPWs, the main event is getting their problem solved and their applications written.

But for responders, beyond the pure altruism of helping out a fellow perler, there are often other reasons for their interest in the OPs problem.

For me, helping you do that--if I can/have--is the side event. The main event is finding the best ways to solve particular problems. For some problems, fork (where available) is the right solution; for some it is threading; for some a select-type mechanism is appropriate. But the problem is that these are too often seen as all-or-nothing alternatives to each other.

And that's where the 'frameworks'--and my distaste for them--come in. Things like POE, AnyEvent, Coro, Parallel::ForkManager; Thread::Pool et al. all try to "simplify" the use of what are, at their hearts, very simple programming constructs. fork, select async etc. But in those attempts to simplify, they fail dismally.

By wrapping over the basic constructs in these huge, unwieldy APIs, they not only trade the small-but-required learning curves of those basic constructs for the far larger learning curves of the own APIs; they also dictate the architecture of the entire applications and application suites in the process. Thus forcing the abandonment of the other basic constructs in the process. And so you end up with one-size (or rather, one-hammer) force-fits-all solutions. They have to be all fork; all select or all threads, because that is what the frameworks expect and dictate. And that's a nonsense.

There is no good reason not to use select within a fork or a thread. Or threads with a fork. Or fork or thread within a select loop. And if you avoid the overarching frameworks built around these basic constructs, and simply learn to use the basic constructs themselves, then choosing the right tool for each particular job is easy.

That's why I've resolutely avoided publishing any, of the many, "threading framework" modules I've written. Because once you move beyond the "poster child" application, into trying to define an API to cover a wide range of typically messy, general purpose problems, the APIs become far more complicated and unwieldy than the basic:

async { while( my $workItem = $Q->dequeue ) { ... } };

they are intended to "simplify" through encapsulation. And along the way, usually preclude the flexibility of using fork or select where that is appropriate.

It's not that those other techniques can't be used within a threading framework. It's just that using them then requires detailed knowledge of the internals of the framework to make them play well together. So then you get into the game of writing wrappers around popular modules like IO::Socket, LWP, etc. The whole thing snowballs and you end up with a huge, fragile and interdependent morass of modules that need intimate knowledge of the framework internals to maintain.

Ie. Exactly what you see when you search for POE::* or AnyEvent::* or Coro::* or Thread::*.

Even if I am capable of writing & maintaining such a suite of complex interdependent modules, I've no desire to have others dependant upon my ability and interest in maintaining them in the long term. But even more important to me, is that I am philosophically opposed to creating such monocultures.

The (my) bottom line is that it would be far better for people to learn to use the basic constructs of fork & exec, select & sysread, async & enqueue()/dequeue() and apply them as required; than to try and skip the (relatively shallow) initial learning curves of those basic constructs by using "a framework", and then finding themselves locked in to monoculture and having to force-fit every aspect of their applications to its limitations and dogmas.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.