ginseng has asked for the wisdom of the Perl Monks concerning the following question:

A friend of mine seems to think he'd like to be able to ping roughly 1,000 hosts every minute, parse the data, and store current ping times into a database. He asked me for help with the "ping 1,000 hosts every minute" part, and we came up with a massively forked ping server.

The prototype concept is that a list of hosts is stored in a text file. A perl script reads in the hosts list, splits it up among a given number of forked processes, and each process gets to work on pinging. All results are appending into a results file for now.

In essence, I seem to be having trouble coding a fork routine that lives within a loop. Everything grows exponentially. For instance, a file with 10 hosts in it ends up with 212 lines in the result file - the 10 hosts are pinged a total of 212 times. I envisioned that once is enough ;)

My question is, how can I create a variable number of child processes, each chewing on a portion of the data? My code fails at that.

So here's my code. It works great, but not to spec. It is also really bad code, may have blatant errors in implementation, and may be poorly structured. It certainly has not evolved to the point where it waits for child processes to end - a very bad habit, I'm sure. My excuse is that I'm trying (and failing) to prove a concept. My apologies are offered in advance.

#!/usr/bin/perl -w use strict; my @hosts = <>; # list of hosts to ping my $depth = 5; # number of children my $size = 800; # packet size my @pid; # store child pids here open (OUT, ">>results"); for (0..$depth - 1) { $pid[$_] = fork; if ($pid[$_]) { print "$_\n"; # for debugging } else { foreach my $host (splice(@hosts, $_ * (scalar @hosts)/$depth, ($_ ++ 1) * (scalar @hosts)/$depth)) { chomp $host; my @result = `ping -c 1 -s $size $host`; $result[1] =~ s/^.*=//mg; # strip the data down to $result[1] =~ s/ ms.*$//mg; # decimal miliseconds print OUT "$host\t$result[1]"; } } } close OUT;
Here's some characteristics of the output:
su-2.04# ./pingfork.pl testping  | sort | uniq -c
  16 0
  16 1
  16 2
  16 3
  16 4
su-2.04# cat results | wc -l
     212

Replies are listed 'Best First'.
Re: A plethora of forks
by blakem (Monsignor) on Aug 31, 2001 at 09:35 UTC
    The first thing that sticks out is your children are not exiting properly. Once they finish their job, they go back into the foreach loop and fork off new processes as if they were the parent. This causes numerous pings for each host. First thing to do then, is to add an exit call right after the child code...
    for ( [snip] ) { $pid[$_] = fork; if ($pid[$_]) { [snip] } else { # ping some hosts exit; # <-- important that your children exit... } }

    -Blake

      Wow. I looked the problem in the face, knew pretty well that the children weren't exiting, and never even thought about the fact I might have to tell them to exit.

      Thank you. That gets me so much closer.

Use 'fping'
by Rippa (Novice) on Aug 31, 2001 at 18:00 UTC
    If I were you, I would write a Perl wrapper around 'fping' - you pass it a bunch of hosts and it will ping them in parallel without you having to manage the forking. It should be on Freshmeat. Good luck....
Re: A plethora of forks
by marcus (Scribe) on Aug 31, 2001 at 18:07 UTC
    Hi..

    Wouldn't it be more sensible to use the native Net::Ping? Then you wouldn't have to deal with forking at all :)

    Cheers

    qw[marcus]

      With Net::Ping, 1 is returned if the host is reachable and 0 if it is not. It doesn't actually ever return the ping time, so it's not useful for determining "how far away" the host is.

      Also, Net::Ping will happily block will it's trying to reach the target host, so even if we wanted only to know if the host is up, we'd still have to fork.

      You're right, though, about turning to CPAN first, and I agree. Unfortunately, I tried it and found that it wouldn't do what was needed.

(MeowChow) Re: A plethora of forks
by MeowChow (Vicar) on Aug 31, 2001 at 22:52 UTC