in reply to how can N client.pl connect to one deamon serving mysql

thanx to saxman!
i modified the deamon.pl to:

#!/usr/bin/perl -w

use strict;
use warnings;

use IO::Socket;
use ResourcePool;
use ResourcePool::Factory::DBI;
use ResourcePool::Command::DBI::Execute;

# Initialize parameter ------------------------------------------------
my $port = 6000;
my $dsn = "DBI:mysql:DBname;DBserver;3306";
my $username = "xxx";
my $passwd = "yyy";

# Initialize Ressource ------------------------------------------------
my $factory = ResourcePool::Factory::DBI->new(
      $dsn,
      $username,
      $passwd);
my $pool = ResourcePool->new($factory, MaxTry => 3);

# Initialize Socket ------------------------------------------------
my $sock = new IO::Socket::INET(
      LocalHost => 'localhost',
      LocalPort => $port,
      Proto => 'tcp',
      Listen => SOMAXCONN,
      Reuse => 1);
$sock or die "no socket :$!";

my($new_sock, $c_addr, $buf);
while(1==1)
{
   while (($new_sock, $c_addr) = $sock->accept())
   {
      my ($client_port, $c_ip) =sockaddr_in($c_addr);
      my $client_ipnum = inet_ntoa($c_ip);
      my $client_host =gethostbyaddr($c_ip, AF_INET);
      my $code = '';
      while (defined ($buf = <$new_sock>))
      {
         $code .= $buf;
      }
      my $cmd = ResourcePool::Command::DBI::Execute->new();
      $pool->execute($cmd, $code);
   }
}
exit;

__END__


Also i used a skript which forked 100 instances of the client.pl each pushing a ~7k INSERT statement to the deamon.pl; this worked fine - all 7000 entries arrived at the DB server and were processed.

it took around 5 secs for the database to execute all statements. that brought me to the following problem:

if there are a lot of client scripts running, the queue will fill up and the clients will keep waiting for the deamon if its queue is full. so the traffic-problem i tried to solve just shiftet to a local connection bottleneck... :-(

is there a possibility to do the socket stuff multithreaded?? each thread with its own socket but using the same sql-queue???
  • Comment on Re: how can N client.pl connect to one deamon serving mysql

Replies are listed 'Best First'.
Re^2: how can N client.pl connect to one deamon serving mysql
by saxman25 (Initiate) on Feb 24, 2006 at 17:04 UTC
    if you
    - just send 1 statement at a time and
    - then disconnect and reconnect for the next one,
    - each transmission terminate by $EOL
    you could try this deamon, which seems to work for several hundred parallel client attempts. (i brute forced it up to 700..800x8kB - took 1.5 min, but the first 500 went through within 20..30sec or so - but with just 100 at a time youre done in about 10 sec. so depending on your average and peek load you have to monitor the runtime of the clients. just do several scenarios for your use and see if there is still a bottleneck...) :


    #!/usr/bin/perl -Tw

    use strict;
    use warnings;
    use IO::Socket;
    use Net::hostent;
    use ResourcePool;
    use ResourcePool::Factory::DBI;
    use ResourcePool::Command::DBI::Execute;

    my $EOL = "\015\012";
    sub spawn;
    my $port = 6000;
    my $proto = getprotobyname('tcp');
    my $dsn = "DBI:mysql:DB;HOST;3306";
    my $username = "xxx";
    my $passwd = "yyy";

    my $factory = ResourcePool::Factory::DBI->new($dsn,$username,$passwd);
    my $pool = ResourcePool->new($factory, MaxTry => 3);

    socket(Server, PF_INET, SOCK_STREAM, $proto) || die "socket: $!";
    setsockopt(Server, SOL_SOCKET, SO_REUSEADDR, pack("l", 1)) || die "setsockopt: $!";
    bind(Server, sockaddr_in($port, INADDR_ANY)) || die "bind: $!";
    listen(Server,SOMAXCONN) || die "listen: $!";

    my $waitedpid = 0;
    my $paddr;
    $SIG{CHLD} = 'IGNORE';

    for ($waitedpid = 0; ($paddr = accept(CLIENT,Server)) || $waitedpid; $waitedpid = 0, close CLIENT)
    {
       next if $waitedpid and not $paddr;
       my($port,$iaddr) = sockaddr_in($paddr);
       my $name = gethostbyaddr($iaddr,AF_INET);

       spawn sub {
          $|=1;
          my $line; my $clientdata = '';
          for (;;)
          {
             undef $!;
             unless (defined($line= <> ))
             {
                die $! if $!;
                last; # reached EOF
             }
             $clientdata .= $line;
             last if eof;
          }
          my $cmd = ResourcePool::Command::DBI::Execute->new();
       $pool->execute($cmd, $clientdata);
       # and here may be a possibility to send data back to the client (f.i. lastinsertid)
       # <> = "place return data here $EOL";
       };
    }

    sub spawn
    {
       my $coderef = shift;
       unless (@_ == 0 && $coderef && ref($coderef) eq 'CODE')
       {confess "usage: spawn CODEREF";}
       my $pid;
       if (!defined($pid = fork))
       {return;} elsif ($pid) {return; # I'm the parent}
       # else I'm the child -- go spawn
       open(STDIN, "<&CLIENT") || die "can't dup client to stdin";
       open(STDOUT, ">&CLIENT") || die "can't dup client to stdout";
       exit &$coderef();
    }
      cool! thanx a lot!!

      now i go and try to implement the bidirectional communication. seems to work so far... *smile*