in reply to Re^4: How to maintain a persistent connection over sockets?
in thread How to maintain a persistent connection over sockets?
That code is remarkable! Also, unfortunately badly wrong, but I'll get back to that.
You have succeeded in writing a multi-tasking server without using any form of multi-tasking. Neither threading, nor forks, nor polling; nor an event loop!
It is utterly, utterly amazing. It took me quite a while to understand just how it achieved that. I now understand your thread title!
It is even more amazing that it achieves the throughput that it does; but is unsurprising that it is not meeting with your expectations.
Your server is resolutely single tasking. And it is also quite difficult to explain how it manages to give the appearance (and indeed, the actions) of being multi-tasking, in terms of the code alone, so I'm going to resort to an analogy.
How can you conduct two (or more) 'concurrent' conversations using one phone that has neither call-waiting; nor conferencing facilities?
The solution is to ask the other parties to disconnect and redial after each snippet of conversation. One person rings, you say "Hello"; they hang up and redial; when you pick up they reply; then hang up and re-dial; this time when you pick up, you reply; and they hang-up and redial; and so on until the conversation is complete.
And if two or more people follow this procedure, then you will be able to hold 'simultaneous' conversations with all of them. They'll just be very slow and disjointed conversations.
That is exactly analogous to how your server is "working".
I am truly surprised at how you arrived at this solution; and totally amazed at how efficiently it actually works. I guess it personifies the old adage about computers being able to do everything very, very quickly; including the wrong thing :)
Of course, it is unsustainable for an application such as yours. You will need to use some form of multi-tasking.
This comes in (essentially) 4 forms with the following traits:
- Event (select) loop.
Ostensibly simple; lightweight, and efficient.
The downside is that all state is global and all communications from all clients go through a single point.
My analogy is having a single telephone and receptionist who has respond to every call and perform all work to satisfy all inbound queries and relay all outbound information.
Works well if the inbound queries can be answered immediately with little effort, but falls down when answering a query requires more effort.
Either every other caller has to wait while the receptionist resolves each query, no matter how long it takes; or the receptionist has to keep interrupting her efforts to resolve the query in order to service other callers.
The first approach means that many clients will wait a long time, even if their queries are fast, whenever a hard to resolve query gets in before them.
The second approach means that long queries take even longer, because the work effort to resolve it keeps getting interrupted by new callers.
- Cooroutines.
I won't discuss this much as I consider it a retrograde step. Like going back in time to Windows 3.0. Only works if everyone cooperates; and they usually do not.
- Multi-processing (forking).
Can be relatively efficient, even for long queries, because each caller gets their own process to respond to them. The downside is, that responder cannot easily communicate with the receptionist, or other responders.
Falls down completely for write to the shared data, because the child process cannot modify the parents copy.
Like having a modern automated switchboard where each new caller is routed directly to the next available agent. The trouble is, each agent sit isolated in their own room with only a copy of the data for reference. They can answer read-only queries, but cannot modify the data that the other agents see. And any nodifications they do make cannot be seen by the other agents.
- Multi-threading.
Similar to the above, in that each caller gets their own, dedicated agent, but now all the agents are in the same room and can easily communicate between themselves. They can all make modifications to the shared data; and all can be aware of the modifications made by others.
The downside -- for a pure Perl, iThreaded implementation -- is that the shared data is (effectively) duplicated for each concurrent client. That makes for extra memory use by the shared data and for slow(ish) inter-agent communications.
The upside (of iThreads) is that only that data that needs to be shared is, and locking is simple (if not exactly fast); which makes it far easier to ensure that the agents don't trample on each others state accidentally and removes most if not all the potential for the classical threading nasties.
Perl threading is far simpler than tradition, all state shared threading. The penalty you pay for that increased simplicity is in memory and performance.
My preferred solution for your application would be a combination of two of the four. Specifically, I would run a single phone&receptionist (a select loop) within a thread. That select loop would take care of all the communications with the clients, but would hand off queries to a pool of agents (work threads).
That allows the receptionist to respond immediately to new callers and inbound queries and modification requests from existing clients; whilst the pool of agents (work threads) take care of the doing the actual work. The pool can be tailored (scaled) to fit the available hardware (number of cores; amount of memory), on a case by case basis; whilst being able to both reference and make modifications to the shared data.
IMO, this will deliver the best combinations of responsiveness and functionality for your scenario.
Give me a few days and I'll get back to you with demonstrations of the 3 main candidates plus my preferred hybrid solution.