Wow. Just when I thought it couldn't get any better than rob_au's code (which looked great), you post this! I'm a little confused how this allows concurrent pings using threads when fork() doesn't appear to, but it's so much better than what I'm using now I just don't care.

Have just done a run of about 4500 machines. Timings:

That's about 26 times faster. I think I want to have your children ;)

And I think Ninthwave nearly wet himself as it's going to benefit a script he's writing at the moment to scan 14000+ machines!

Cheers,

Gordon.

P.S. I don't really actually want to have your children. Biological impossibility. You knew that, right? :)

* Combined the "while(1) with the next 2 lines
* Added a result q , and print of results
* Yield proc on successful ping
* Changed ping packet size so it doesnt get blocked by "Nachi" virus blockers
* Added a "real" range of 256 IP's to ping.

Here is the result:

use strict;
use threads;
use Thread::Queue;
use Net::Ping;

my $ThreadCount = 12;
my @Threads;
my $ServerQueue = Thread::Queue->new;
my $ResultQ     = Thread::Queue->new;

my @Servers;

push @Servers, qq(4.2.2.$_) for (1..254);

foreach my $Item (@Servers) {
   $ServerQueue->enqueue($Item);
}

for (1..$ThreadCount) {
   my $Thread = threads->new(\&Process, $ServerQueue);
   push (@Threads, $Thread);
}

foreach my $Thread (@Threads) {
   $Thread->join();
}

print "\n--results---\n";
while (my $target = $ResultQ->dequeue_nb){
      print "  $target\n";
}
#####################################
sub Process {
   my $ServerQueue = shift;
   my $pingobj = Net::Ping->new("icmp",5,100);

   while (my $Server = $ServerQueue->dequeue_nb()) {
       if ($pingobj->ping($Server)) {
           #ping was good
            $ResultQ->enqueue ($Server);
            threads->yield();
       } else {
           #ping was bad
       }
   }
}
[download]

I'd agree with those changes -- I've done pretty much exactly the same when shoe-horning it into one of my scripts.

I hadn't thought of putting a threads->yield in there, and I guess that makes the code a bit more friendly -- it doesn't appear to impact performance at all so I've added that too.

One other thing I've done is to check how many machines are to be pinged, and dropped the number of threads if it's a small number (i.e. no point spawning 20 threads to ping five machines)

Thanks everybody. Since I have 5000+ servers to deal with, I have to be pretty creative with everything I do. Almost all the code I write has to be based on threads, if I hope to get anything done in a reasonable time.

As for the changes NetWallah made above, they do make sense and I should probably have included them. The real code I use in production is a bit different:

IP Addresses are pulled from a database. Since I don't want to use up a lot of memory putting all 5000+ entries into an array, I actually spawn the threads first, and then slowly feed entries into the Queue as I read them from the database.

I needed a "while (1)" because there are times in the thread's life when the queue is empty but it should wait rather than exiting, such as when it's first created. My production code has a shared flag indicating whether all entries have been loaded into the queue, and then each thread will "sleep 1; next;" or "last;" based on the status of that shared flag.

Also, rather than have a "result" array in memory holding thousands of results to print afterwards, each thread maintains an open connection to the database during its life, and uses it to store the results.

Instead of using the shared flag would it be better to use Thread::Queue::Monitored as I have found that the array loading is pretty much a drag and am thinking of spawning the threads, loading the ips directly to the queues and at the end of the queing fire off a series of exit commands equal to the threads waiting.

I am trying it now and hopefully have some code soon as an example, but was just curious if you had tried it?

Update:
It appears that Thread::Queue::Monitored only sets up one thread to Monitor the queue which I am going to try to use as the back end process to get the results of the pings instead of trying to shove them into the array. Have the Monitored Queue do the db processing.

"No matter where you go, there you are." BB

This is the code I have come up with. This is for a Win32 machine and I am using Win32::ODBC with MySQL. I have not coded it yet to be database independent but that will happen as I update the code but the thread processing works fine and I am able to scan a class A fairly efficiently. There seems to be a small memory leak in it but it does not affect the scan adversely. Memory grows very slowly over a large period of time. With the command line options though you can have the script scan ranges from a batch file or bash script exit and than do another range minimising the affects of the leak.

Read more... (11 kB)

The Pod is not complete as I am still working on the script. I have to remove references to cab extract as I am doing that process in another script and I have to describe all command line options though it is fairly evident from the code what the options do.

One major note is we were getting responses where the ping replied by gethost returned null. I created a unique computer name based off that and am currently working on a clean up section that tries to identify the invidual devices.

Any additional clean up or efficiency suggestions would be appreciated.

"No matter where you go, there you are." BB

Thank you.
I didn't wet my pants but I was very impressed by the simple elegance of this and the thread pragma.

Update much later after using ithreads for awhile

And very annoyed by the memory leaks and pain of using complex data structures anywhere near threads. Oh well neither here or there thank you everyone for the info.