I wrote a script to log ping latencies for a bunch of hosts on our network. It works well for the most part, but there's a bug that rears it's head sporadically, and I haven't been able to track it down.

Here's the meat of the code (some vars are defined earlier in the script):

my $p = Net::Ping->new("icmp",$timeout,32);
while (1) {
   foreach my $host (@hosts) {
      my @times;
      for (1..4) {
         my $t0 = [gettimeofday];
         my $alive = $p->ping($host);
         my $t1 = tv_interval ($t0, [gettimeofday]);
         push @times, 
            $alive ?             # was ping successful?
            int($t1 * 1000) :    # if so, record interval
            $timeout * 1000;     # if not, record timeout val
      }
      my ($sec, $min, $hr, $day, $month, $year) = (localtime)[0..5];
      my $timestamp = sprintf("%04d-%02d-%02d,%02d:%02d:%02d",
         $year + 1900, $month + 1, $day, $hr, $min, $sec);
      print "$timestamp,$host,", join ( ',', @times ), "\n";
   }
   sleep $yawn;
}
[download]

This produces output that looks like:

2003-08-15,17:02:19,10.10.10.50,62,46,62,46
2003-08-15,17:02:20,10.10.10.60,109,46,62,62
2003-08-15,17:02:20,10.10.10.70,93,62,46,62
2003-08-15,17:02:20,10.10.10.80,109,62,46,62
2003-08-15,17:02:21,10.10.10.90,93,62,62,46
2003-08-15,17:02:23,10.10.10.50,46,62,46,62
2003-08-15,17:02:23,10.10.10.60,46,62,62,62
2003-08-15,17:02:23,10.10.10.70,46,62,46,62
2003-08-15,17:02:23,10.10.10.80,62,46,62,46
2003-08-15,17:02:24,10.10.10.90,62,62,46,62
[download]

This is handy for importing into a spreadsheet or database, averaging the ping times, etc.

Now, the problem. If I let this run for several minutes, eventually I get output like this:

2003-08-15,17:11:04,10.10.10.50,46,62,46,46
2003-08-15,17:11:04,10.10.10.60,62,62,46,62
2003-08-15,17:11:05,10.10.10.70,62,62,46,62
2003-08-15,17:11:05,10.10.10.80,46,62,62,46
2003-08-15,17:11:05,10.10.10.90,62,46,62,62
2003-08-15,17:11:07,10.10.10.60,2003-08-15,17:11:08,10.10.10.70,2003-0
+8-15,17:11:08,10.10.10.80,2003-08-15,17:11:08,10.10.10.90,2003-08-15,
+17:11:11,10.10.10.50,2003-08-15,17:11:11,10.10.10.60,2003-08-15,17:11
+:11,10.10.10.70,2003-08-15,17:11:11,10.10.10.80,2003-08-15,17:11:12,1
+0.10.10.90,
[download]

In case that lost or gained some formatting during posting, the problem is that the code eventually skips one host, then stops printing the @times values and the following newline for all successive iterations. The code continues running and logging, but appends only the date, time, and hostname. This last line grows longer ad infinitum.

I'm only just starting to troubleshoot this, but I need to get it running pretty quickly, so I thought I'd open it up to the Monks as well. Is there a flaw in the code, or is there a network condition that could cause this oddness?

Thanks!