ablmf has asked for the wisdom of the Perl Monks concerning the following question:

I am using IO:SELECT. All the examples I've read is like this. 
while(@ready = $sel->can_read) {
    foreach $fh (@ready) {
        if($fh == $lsn) {
            # Create a new socket
        }
        else {
            # Process socket
        }
    }
}

[download]
But if I have to do some Remote Database operation, the whole program will be blocked.
What can I do to avoid this? I'm using select becasue there would be 200 connections. It's clearly that I can not create 200 threads or processes

Replies are listed 'Best First'.
Re: a IO:SELECT problem
by Corion (Patriarch) on Jun 25, 2006 at 16:54 UTC

    select in Perl and Unix only works with files and TCP and Unix domain sockets. On Win32 Perl, select only works with TCP sockets. If you want to run multiple database operations concurrently, you will have to spawn a thread or a process for every client connected or somehow pool your clients requests. If you have not already done so, look at what Apache::DBI does to pool its DBI database connections.

[reply]
[d/l]
[select]
Re: a IO:SELECT problem
by sgifford (Prior) on Jun 26, 2006 at 01:55 UTC
    Interesting question. My first thought is: What's the big deal about 200 processes or threads? That would almost certainly be the easiest solution.

    Assuming there's some good reason why that won't work, what you would need to be able to do is asynchronous database operations. Either a way to get a callback or a signal when results are ready, an implementation that guaranteed that some filehandle would be selectable only when there's results to read, or at least something you could poll with. You could look at your database's driver options to see if this is possible; if only the C version of the driver supports it, you could use Inline::C to talk to it.

    Another option would be to proxy the results through something that would send you some kind of notification, or only make a filehandle selectable when results were ready. You could use some kind of RPC interface, like SOAP, or extend something like DBD::pNet to give it these capabilities.

    But those are much harder than managing 200 threads or processes. :-)


    --
    sgifford's Web page
[reply]
[d/l]
[select]

      Thanks a lot for all your help.

      But won't using 200 processes or thread be too resource consumable comparing to use select or poll?

      If it would considerable low down the performance of the program, it would be unacceptable. My program will running on a linux server.

      By the way, I am using mysql. Does it provide asynchronous operation?

[reply]
        It depends on what the processes are doing; if they're mostly sleeping while waiting for results to become available, resource use will be minimal. If they're all trying to do a lot of work at the same time, they may overwhelm the CPU, and you'll have to be careful to throttle them a bit. Parallel::ForkManager can be helpful with that sort of thing.

        I just glanced through the MySQL docs, and I didn't see any kind of asynchronous API. You may get some help on the MySQL mailing list; there might be a clever hack that will allow this.


        --
        sgifford's Web page
[reply]

        Unless your MySQl server is running distributed, or on a machine that has dozens of processors, your 200 'concurrent' queries are going to be seriealised anyway.

        It might be better to start a (small, 2 or 3) pool of DB worker threads in your application, and queue the queiries and results to/from the server. However you choose to do it, only 1 or 2 of your 200 queries are actually going to run concurrently, and the 200 threads/processes that issue those queries will all be waiting for their results anyway.

        It's really quite hard to see what you hope to achieve by issuing concurrent queries from a client process?

        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

Back to Seekers of Perl Wisdom