Here is the parred down code and where the error occurs:

use strict;
use IO::Socket;
use threads;
use threads::shared;

my $numClients = shift;
my %thread_status : shared;
my $server_status : shared = 'threading';
$| = 1;
my $screen_output : shared = 1;

my $HOST = 'cdt-2x4w227x';
my $PORT = 7001;

# Launch clients in their own thread
print "Creating $numClients clients\n";
for (my $i=0; $i<$numClients; $i++) {
    my ($retries,$socket);

    $socket = IO::Socket::INET->new("$HOST:$PORT") || print "Socket cr
+eation error: $!\n";
    while (!$socket) {
        print "Retrying socket connection for client $i\n";
        $retries++;
        if ($retries > 5) {
            print "Exceeded retries while launching client $i\n";
            last;
        }
        sleep 1;
        $socket = IO::Socket::INET->new("$HOST:$PORT") || print "Socke
+t creation error on retry $retries: $!\n";
    }
    if ($socket) { my $t = threads->new(\&handle_client,$socket,$i)->d
+etach } else { print " Error creating socket: $i\n" }
    print ".";
}
print "Done creating clients\n";

sleep 1;
$server_status = 'ready';

print "Waiting for client threads to finish\n";
while (my @keys = keys %thread_status) {
    my $count = @keys;
    print &time_stamp," Waiting for [$count] threads from [@keys]\n";
    sleep 1;
}
sleep 1;

#<------------------------------------------------------------ handle_
+client -->
sub handle_client {
    my ($socket,$clientNum) = @_;
    my $line;
    my $tid = threads->self->tid;
    my $timeout = 2;

    $thread_status{$clientNum} = $clientNum;

    while ($server_status eq 'threading') { sleep 1 }

    #VINI
    print " $clientNum -> VINI\n" unless (!$screen_output);
    if ($socket) { $socket->send("VINI\n") || $socket->close && undef 
+$socket && return }
    ^^^^^^ FAILS ON SEND ^^^^^^^
    $line = <$socket> if ($socket);
    print " $clientNum <- $line" unless (!$screen_output);
    sleep rand($timeout);

    lock %thread_status;
    delete $thread_status{$clientNum};
    $socket->close;
    threads->self->yield;
    return;
}
[download]

Error is: thread failed to start: Can't call method "send" without a package or object reference at thread_test.pl line 60

So it looks like you're right about the timeout. So how would one handle and recover from such a time out? Thanks!

I chopped down your code quite a bit, and also created a multi-threading server. I tried with 100 socket, and don’t see any problem.

Failed to go 200, the problem is low virtual memory, any way, the code itself works.



server.pl:
use IO::Socket::INET;
use threads;

use strict;

my $server = new IO::Socket::INET(Proto => "tcp", LocalPort => 7001, L
+isten => 5)
          || die "failed to establish socket\n";
while (1) {
    my $client = $server->accept;
    threads->create(\&client_handle, $client);
}

sub client_handle {
    my $client = shift;
    while (1) {
        my $line = <$client>;
        chomp $line;
        print $client $line + 1, "\n";
    }
}

client.pl:
use strict;
use IO::Socket;
use threads;
use threads::shared;

my $numClients = shift;
my %thread_status : shared;
my $server_status : shared = 'threading';
$| = 1;

print "Creating $numClients clients\n";
for (my $i=0; $i<$numClients; $i++) {
    my $socket = IO::Socket::INET->new(Proto => "tcp", PeerAddr => "lo
+calhost", PeerPort => 7001) || print "Socket creation error: $!\n";
    threads->new(\&handle_client,$socket,$i)->detach;
    print ".";
}
print "\nAll Created\n";

sleep 1;
$server_status = 'ready';

<STDIN>;

#<------------------------------------------------------------ handle_
+client -->
sub handle_client {
    my ($socket,$clientNum) = @_;
 
    while ($server_status eq 'threading') { sleep 1 }
    
    print $socket "1\n";
    while (1) {
        my $line = <$socket>;
        chomp $line;
        print "$clientNum <<== $line\n";
        print $socket $line + 1, "\n";
        sleep 1;
    }
    return;
}
[download]

I was able to get about 4,000 SMTP transactions per minute from a local network by 'pre-sporking' threads to sit and listen on sockets:

use strict;
use threads;
use IO::Socket;

# the port to listen on:
my $PORT = 8989;

# the number of threads to pre-spork:
my $INITHREADS = 5;

my $handle = IO::Socket::INET->new(
    LocalPort => $PORT,
    MultiHomed => 1,
    Type => SOCK_STREAM,
    Reuse => 1,
    Listen => 10) or die "Cannot open port $PORT: $!\n";
$handle->autoflush(1);

for(1..$INITHREADS)
    { my $thr = threads->new(\&process)->detach(); }

sleep(); # The parent thread sleeps forever. . .
# You can potentially change this perma-sleep to
# have the parent thread do dynamic thread management
# but only if you don't detach() them.

sub process {
    while(my $socklet = $handle->accept()) {
    # if you want to capture the IP address:
    # my $peerhost = $socklet->peerhost();
    # Here you can do your stuff
    # I have the server talk to the client
    # via print $socklet and while(<$socklet>)
    }
}
[download]

I kept that code for future testing on the server side.

However, right now, it's the client side that is having the problem. The server seems to take anything I can throw at it so far with the exception that some connection attempts to it fail, probably because it's just too busy?

I was wondering too if there was some sort of file handle limit I was running in to. I have not been able to find out what it might be using perl 5.8.0 from activestate under win2k. I see a lot of talk about 256 but can't verify it. :-(

yeah, it seems ActiveState 5.8.0 under WinXP, only 256 threads can be created. If Try to start more than that the program exits, no error, no warning. . . .:

use strict;
use threads;

my $num_threads = 0;

for(1..257) {
    my $thr = threads->new(\&threader);
    $num_threads++;
    print "$num_threads started\r";
}

print "$num_threads running\n";
sleep();

sub threader {
    sleep();
}
[download]

use strict;
use IO::Socket;
use threads;
use threads::shared;

my $numClients = shift;
my $all_created : shared = 0;
$| = 1;

threads->create(\&server);
print "Creating $numClients clients...\n";
for (1..$numClients) {
    my $socket = IO::Socket::INET->new(Proto => "tcp", PeerAddr => "lo
+calhost", PeerPort => 7001) || print "Socket creation error: $!\n";
    threads->new(\&client, $socket, $_)->detach;
    print ".";
}
print "\nAll Created\n";
sleep 1;
$all_created = 1;
<STDIN>;

sub server {
    my $server = new IO::Socket::INET(Proto => "tcp", LocalPort => 700
+1, Listen => 5)
          || die "failed to establish socket\n";
    while (1) {
        my $client = $server->accept;
        threads->create(\&client_handle, $client);
        threads->yield;
    }
}

sub client_handle {
    my $client = shift;
    while (<$client>) {
        chomp;
        print $client $_ + 1, "\n";
        threads->yield;
    }
}

sub client {
    my ($socket,$id) = @_;

    sleep 1 while (!$all_created);
    
    print $socket "1\n";
    while (<$socket>) {
        chomp;
        print "$id <<== $_\n";
        print $socket $_ + 1, "\n";
        threads->yield;
        sleep 5;
    }
    return;
}
[download]

pg,

I have done a whole bunch of testing. All with perl 5.8 under both linux and win2k,usually one as the client and the other as the server.

I've used my code, and your first code above. You're right, everything is working fine at 100. The hard stop seems to be at 110. Or actually it might be 113.

With win2k as the server, I get all kinds of 'connection refused' errors. Of course then I get all kinds of errors. Sometimes it has seemed to help to set Listen=>50 or something higher anyway. .

With linux as the server and generating 120 clients from win2k, all connections seem to occur, however only the first 110 clients ever talk. I read some where that under windows there is a main thread and a thread for 'use threads' and a thread for 'use threads::shared'. Those 3 plus the 110 clients that talk = 113.

Throughout the other various test I have run, the numbers 110 or 113 keep cropping up as the limits. So apparently there is something built in that I cannot control as far as the number of simultaneous connections goes.

Of course this is very discouraging as I will now need to look around for another language to use to get a stress tester built. It pains me as I promote perl around here and now I'm not looking too good on this one. :-(

If you have any other ideas that I can try, please let me know. I have even tried using arrays of IO::Select objects without threading, no go.

Thanks for all your help!!

Joel

You don't need to create more than one server socket if you Reuse

you can:


my $server = new IO::Socket::INET(
Proto => "tcp",
LocalPort => 7001,
Listen => 5,
Reuse => 1) || die "failed to establish socket\n";
[download]

And then accept in the thread.

I believe the "thread failed to start" error is generally followed by an error with a line number. Please correct me if I'm wrong and if not, please post the whole error.

smiles (for everyone)

Doh, sorry. See reply to first responder.....