# Like tcp protocol, but with many hosts
    $p = Net::Ping->new("syn");
    $p->{port_num} = getservbyname("http", "tcp");
    foreach $host (@host_array) {
      $p->ping($host);
    }
    while (($host,$rtt,$ip) = $p->ack) {
      print "HOST: $host [$ip] ACKed in $rtt seconds.\n";
    }
[download]

Thanks Roger. I'm a little concerned I might upset our security people by using Syn scans, but it looks like it could be a good solution so I'll check it out when I'm back in work.

Cheers,

Gordon.

Also be aware that the sending of SYN packets through an interface may require elevated privileges for the user and/or application.

 

perl -le "print+unpack'N',pack'B32','00000000000000000000001010010011'"

Superb, that looks to be exactly what I'm after!

Thanks,

Gordon

use strict;
use threads;
use Thread::Queue;
use Net::Ping;

my $ThreadCount = 6;
my @Threads;
my $ServerQueue = Thread::Queue->new;

# my @Servers = whatever
foreach my $Item (@Servers) { 
    $ServerQueue->enqueue($Item);
}

for (1..$ThreadCount) {
    my $Thread = threads->new(\&Process, $ServerQueue);
    push (@Threads, $Thread);
}

foreach my $Thread (@Threads) {
    $Thread->join();
}

sub Process {
    my $ServerQueue = shift;
    my $pingobj = Net::Ping->new("icmp",5,64);

    while (1) {
        my $Server = $ServerQueue->dequeue_nb();
        last if (! $Server);

        if ($pingobj->ping($Server)) {
            #ping was good
        } else {
            #ping was bad
        }
    }
}
[download]

Wow. Just when I thought it couldn't get any better than rob_au's code (which looked great), you post this! I'm a little confused how this allows concurrent pings using threads when fork() doesn't appear to, but it's so much better than what I'm using now I just don't care.

Have just done a run of about 4500 machines. Timings:

• Sequential: 4464 seconds
• Threaded: 172 seconds

That's about 26 times faster. I think I want to have your children ;)

And I think Ninthwave nearly wet himself as it's going to benefit a script he's writing at the moment to scan 14000+ machines!

Cheers,

Gordon.

P.S. I don't really actually want to have your children. Biological impossibility. You knew that, right? :)

Some minor cosmetic and functional tweaks to your great code:

* Combined the "while(1) with the next 2 lines
* Added a result q , and print of results
* Yield proc on successful ping
* Changed ping packet size so it doesnt get blocked by "Nachi" virus blockers
* Added a "real" range of 256 IP's to ping.

Here is the result:

use strict;
use threads;
use Thread::Queue;
use Net::Ping;

my $ThreadCount = 12;
my @Threads;
my $ServerQueue = Thread::Queue->new;
my $ResultQ     = Thread::Queue->new;

my @Servers;

push @Servers, qq(4.2.2.$_) for (1..254);

foreach my $Item (@Servers) {
   $ServerQueue->enqueue($Item);
}

for (1..$ThreadCount) {
   my $Thread = threads->new(\&Process, $ServerQueue);
   push (@Threads, $Thread);
}

foreach my $Thread (@Threads) {
   $Thread->join();
}

print "\n--results---\n";
while (my $target = $ResultQ->dequeue_nb){
      print "  $target\n";
}
#####################################
sub Process {
   my $ServerQueue = shift;
   my $pingobj = Net::Ping->new("icmp",5,100);

   while (my $Server = $ServerQueue->dequeue_nb()) {
       if ($pingobj->ping($Server)) {
           #ping was good
            $ResultQ->enqueue ($Server);
            threads->yield();
       } else {
           #ping was bad
       }
   }
}
[download]

I'd agree with those changes -- I've done pretty much exactly the same when shoe-horning it into one of my scripts.

I hadn't thought of putting a threads->yield in there, and I guess that makes the code a bit more friendly -- it doesn't appear to impact performance at all so I've added that too.

One other thing I've done is to check how many machines are to be pinged, and dropped the number of threads if it's a small number (i.e. no point spawning 20 threads to ping five machines)

Thanks everybody. Since I have 5000+ servers to deal with, I have to be pretty creative with everything I do. Almost all the code I write has to be based on threads, if I hope to get anything done in a reasonable time.

As for the changes NetWallah made above, they do make sense and I should probably have included them. The real code I use in production is a bit different:

IP Addresses are pulled from a database. Since I don't want to use up a lot of memory putting all 5000+ entries into an array, I actually spawn the threads first, and then slowly feed entries into the Queue as I read them from the database.

I needed a "while (1)" because there are times in the thread's life when the queue is empty but it should wait rather than exiting, such as when it's first created. My production code has a shared flag indicating whether all entries have been loaded into the queue, and then each thread will "sleep 1; next;" or "last;" based on the status of that shared flag.

Also, rather than have a "result" array in memory holding thousands of results to print afterwards, each thread maintains an open connection to the database during its life, and uses it to store the results.

Thank you.
I didn't wet my pants but I was very impressed by the simple elegance of this and the thread pragma.

Update much later after using ithreads for awhile

And very annoyed by the memory leaks and pain of using complex data structures anywhere near threads. Oh well neither here or there thank you everyone for the info.

"No matter where you go, there you are." BB

Well Gordon I can tell you about this at work tomorrow. But that would mean me hovering over your desk and having to listen to another meeting hosted by Rimmer as I am sure you will have at least 12 tomorrow. But it is worth noting on the bottom of the documentation for Net::Ping there is this link to known bugs Multi-Thread Net::Ping It appears that the author of the module agrees with the Roger here.

Some modules to look at:

• POE::Component::Client::Ping
• Win32::PingICMP
• fping

I will look into them more tomorrow morning as you know I need the same functionality.

There is a program, fping, that does simultaneous pings to multiple hosts. It is available for Unix but not for Windows.

The second one works much the same but is much more intellegent and can report on the accumulation of errors. Say that a level one error is 7 failures in 10 minutes and a level two is 12 in fifteen for example. Again, it sends an SNMP trap, but that section could just be cut out.

They are not 'done' yet but work perfectly fine. Just a little rough on the user interface side. I will post them here. Let me know if you have questions. I planned on posting this once it was 'done' but since you have a need.....

Please be advised, this was written for and on Unix and will require slight porting to Win.

Looks interesting, though I have a feeling it may suffer from the same problem as my attempts at using fork() -- namely that fork() is simulated on Win32 using threads, and the Net::Ping ICMP ping method is a blocking operation.

What happens, therefore, is that only one ping runs at a time and the application actually takes longer to complete due to the overhead of fork().

I'll take a closer look at this when I'm in work again (since I have over 3000 machines to play with there, compared to just five here...) to see what happens on Win32.

Thanks for your response.

Gordon.

The first one I posted suffers from performance issues since it forks each time the operation runs. What I would like to know is what you mean by 'Net::Ping ICMP ping method is a blocking operation'? In a forked environment, even if it is faked by threads, you should still have each child able to run on its own, please correct me if this is not so.

The design is that each child runs independantly and when it dies the parent reaps it. Depending on the return status of the child that died the parent either continues on to reap another, or sends and SNMP trap.

The second program on my scratchpad however is much more resource friendly. It preforks a number of child processes which do not exit. The list of destination information is handed out one by one to the child processes and if they fail they are logged to a database via another processes. There is yet another child processes who's only job is to monitor the database of failures and report in whatever you set to be a failure (7 failures in 10 minutes, etc) and clean out the old entries.

Since the child processes all stay running the parent which hands out the IP's logs what time each was handed out and will only hand it out once every 60 seconds. What happens is that due to failures and other latency the program balances itself out over time. This has the effect of spreading out the pings over the 60 second (configurable) interval to lessen the load on the server as well as the network.

Let me know if you have any more questions on this second program. It will probably have much more work to port to a Win platform though.

Have you ever considered running this in cygwin? Is there a *nix box you could put it on? Just because you are checking windows boxes dosen't mean all your tools have to run on one.