hammopau has asked for the wisdom of the Perl Monks concerning the following question:

Hi,

First off, this is my first posting on perlmonks and my first real perl script, so pls be gentle...

The script below is used to run multiple instances of the required command (ping, ssh, any function as reqd, but also uses sleep for testing) in a sub. To launch and manage/limit the instances running at any 1 time, I use fork and waitpid.

The input file is a plain text file containing hostnames. In the full script, the hostname is passed to the sub in question, but as this is not required for testing with sleep I have taken it out.

When I run the script with "for (<IN>)" the script works fine, but this will create a temp array (so I'm told). As the input file/list of hosts will run into the 1000's this will use memory where ideally it shouldnt. If I use "while (<IN>)" (which does not create a temp array/use memory), the script just loops the inputfile. It will run until you kill it, regardless of how many hosts are in the input file.

I'd like to do this cleanly with "while" rather than "for" if possible. Does anyone have any good/clean/noob friendly ideas? It may be easier to bite the bullet & use "for", but I thought I'd ask first...

perl version is 5.8.5

Regards
Paul

## Additional Info ##
Apologies for not including in original post:

The loop only occurs when the "waitpid" function is called. If the max number of runs is not met ($max=2 in example), the while finishes ok.

As such this may be a while/fork or waitpid issue.

#!perl -w use strict; use Data::Dumper; my $prog=$0; my ($estat,$infile,$errfile,$action,$sub,$thishost,$pid,$cpid,$pidstat +,$thiskey); my (%track,%closed); my $max=2; my $rcount=0; my $tcount=0; $infile=$ARGV[0]; open(IN,'<',$infile) || die "cannot open input file $infile \"$!\""; sub _sleep { sleep int(rand(10)); return $?>>8; } sub _waitpid { my $cpid=waitpid(-1,0); my $pidstat=$? >> 8; $closed{$track{$cpid}}=$pidstat; $rcount--; } #here while (<IN>) { #for (<IN>) { chomp; $rcount++; $tcount++; print "Progress: $tcount\r"; $pid = fork; if ($pid==0) { # child close IN; # start call sleep sub my $estat=_sleep; exit $estat; # end call sleep sub # start direct sleep #sleep int(rand(10)); #exit $?>>8; # end direct sleep } else { # parent $track{$pid}=$_; if ( $rcount >= $max ) { # start call waitpid sub _waitpid(); # end call waitpid sub # start direct waitpid #my $cpid=waitpid(-1,0); #my $pidstat=$? >> 8; #$closed{$track{$cpid}}=$pidstat; #$rcount--; # end direct waitpid } else { } } } print "\n\nWaiting for procs to finish...\n"; while ($rcount>0) { _waitpid(); } print "Done\n"; for $thiskey (keys %closed) { print "$thiskey $closed{$thiskey}\n"; } print Dumper(\%closed); close IN; print "End\n";

Replies are listed 'Best First'.
Re: while/fork problem
by shmem (Chancellor) on Jun 12, 2007 at 16:33 UTC
    Replacing for(<IN>) with while(<IN>) works fine. The for() reads the entire file and builds a list to iterate over, while the while() just reads one line at a time. Inside the loop the $_ is just the same. With modification in two places
    #here while (<IN>) { #for (<IN>) { chomp; $rcount++; $tcount++; print "Progress: $tcount\n"; # changed \r to \n $pid = fork; if ($pid==0) { # child close IN; print "child pid $$\n"; # new debug line # start call sleep sub

    your script produces

    Progress: 1 child pid 21079 Progress: 2 child pid 21080 Progress: 3 child pid 21081 Progress: 4 child pid 21082 ...

    so I guess it works as expected. It is definitely a better to use while() if you don't need the entire bunch of lines (e.g. to sort them first), but one line at a time.

    --shmem

    _($_=" "x(1<<5)."?\n".q·/)Oo.  G°\        /
                                  /\_¯/(q    /
    ----------------------------  \__(m.====·.(_("always off the crowd"))."·
    ");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}
      shmem,

      Thanks for the reply. The problem is only encountered when the waitpid function is called when the max number of runs is hit. The sleep is currently used to make sure this happens. Are you still calling the sleep. If you are calling waitpid and the 'while' works ok, it may be a problem with our version of perl.

      Can you pls confirm & thanks again...

      Paul

Re: while/fork problem
by zentara (Cardinal) on Jun 12, 2007 at 17:07 UTC
    so pls be gentle...

    A point for begging for mercy. :-)


    I'm not really a human, but I play one on earth. Cogito ergo sum a bum