Convert application to async

arpx has asked for the wisdom of the Perl Monks concerning the following question:

I got a large mod_perl app, currently running apache event MPM.
but now im looking for ways to convert it to more pure asynchronous perl (maybe with node.js)
The theoretical advantage would be speed and load stability, including avoiding issues with poor thread-safe modules resulting in a extended use of lock algorithms.

My thoughts so far:
- this will require some sort of higher level dispatcher that would handle the async blocks. The dispatcher would sit between the httpd and the application.
- the scale of the program makes it unrealistic to rewrite, so the best way would be to place a lot of yields to the dispatcher in the current application (at regular space and at points of interest where external calls can delay).
- the dispatcher would handle some a execute queue and let some entries tie up objects to wait for (db or file reads) and if not done, push back in the queue.

Haven't got further yet in thoughts of execution..
But ideas and input would be of great help! Maybe there is better approaches..

Cheers,

Fredrik

Comment on Convert application to async

Replies are listed 'Best First'.
Re: Convert application to async by Corion (Patriarch) on Mar 10, 2013 at 16:41 UTC
One "easy" way would be to make the application asynchronous using Coro. Coro basically introduces what other languages call Green Threads to Perl. The drawback is that Coro does some dark magic to swap call stacks around, which may or may not play well with Apache, so you best put that functionality into a separate process. And even then it still employs that dark magic. See Twiggy and Starlet for Coro-based HTTP servers for example.	[reply]
Re^2: Convert application to async by arpx (Acolyte) on Mar 10, 2013 at 22:54 UTC
nice, looks clean and pretty.. will dig into them tomorrow, might fit tx for your hint! :) cheers,	[reply]
Re^3: Convert application to async by karlgoethebier (Abbot) on Mar 11, 2013 at 08:15 UTC
Worth reading: Why Coro?. Regards, Karl ŤThe Crux of the Biscuit is the Apostropheť	[reply]
Re: Convert application to async by Rhandom (Curate) on Mar 11, 2013 at 03:37 UTC
I don't know. But I am interested. What sort of loads does your mod_perl application do? Is it a chat system, a blog, a heavy database query system, a simple web form, a game server? How many people connect to your system - requests per second and concurrent users? How many webservers are simultaneously serving this application (is it on a single box)? Is the database on the same box? Is the database mysql, postgres, a nosql, or some other? Do you have any caching layers (memcache, kyoto, file based)? I'm interested in what aspects of your application have led you to async operations as the solution, even if it is just "to scratch that itch." I think you'll get more comprehensive feedback if you fill in a few more blanks. my @a=qw(random brilliant braindead); print $a[rand(@a)];	[reply]
Re: Convert application to async by locked_user sundialsvc4 (Abbot) on Mar 11, 2013 at 21:26 UTC
I can definitely say that you need to tell us more, much more, about this application ... and about how and why it hurts. It certainly sounds to me like the Perl application, operating as it does “wedged into” an Apache service-process, has what has become a very inappropriate design. But exactly what to do about it, at this point, is not a question easily answered. It’s going to be a compromise decision. The hugely important question is: what compromise to make, and at what technical risk therefore cost. My instincts tell me that what you really have here is a workflow management system, for which the Perl application ought to be ... a user interface. The work is being done by a different set of processes on the same or a different computer. But, since you say that “the scale of the program makes it unrealistic to rewrite,” the matter of making compromise technical decisions appears again. Fact is, you are going to have to “rewrite it” somewhat, and there will be no bright-line guidance with regard to how exactly to do so. If you will, please follow-up on this thread with more detail. The necessary consultative expertise is unquestionably to be found here ... and not particularly in “me.” The task before you is going to require a lot of thought, and not-so-much programming.