Galen has asked for the wisdom of the Perl Monks concerning the following question:

I have a cgi script that takes a selected list of network elements, telnets to them, issues a command and then echo's the response back to a web page. It works great, but I have one last functionality to add that is causing me headaches. When the list of elements is long, I can't afford to do each sequentially, because it takes 30 seconds or so for each element. The user sits there forever waiting for data to be returned. I want to use fork() to solve this problem. Imagine I have 20 elements - I want to telnet out to each of them simultaneously and gather data, and return the data when all 20 are completed. If done simultaneously with fork(), this should only require about 30 seconds, as opposed to 10 minutes if done sequentially.

I'm not sure how to approach this problem. I started by building something very simple - a script to read 3 text files and echo their contents back to the command line:. I stored the filenames in an array in order to mimic the way I read a list of network elements from an array. If I can get this to work, I should be able to use the same logic in my network code. 

#!/usr/bin/perl
# forkit.pl
# This is my training script on fork.  I will attempt to read and echo
+ data
# from three files simultaneously.  The files are: file1, file2, file3

use CGI qw(:standard);

# Here is an array containing the names of the files that I will open

@files = ("file1", "file2", "file3");

# Let's show that we have correctly stored the filenames
print "\nThe files I will be looking at are:\n\n";
foreach (@files) {
print "$_\n";
}
print "\n";

# OK now I will open these files and echo their contents sequentially
# without using fork() ---

foreach (@files) {
open(EP,$_);
print "\nI just opened $_\n";
  while (<EP>) {
    chomp;
    print "Contents of this file: $_\n";
  }
}
print "\n";

# And now, do the same thing using fork()

[download]

I realize that I need to set up a condition so that each fork knows who it is based on the PID and executes appropriately. But, I'm getting some strange behavior. I would have thought it would simply be - foreach file, fork and if you are the child read the file and echo contents. Not so easy though... I am apparently forking more than I should. Also, I'm not even grabbing the contents of the files with the code below. What am I doing wrong? 

foreach (@files) {
open(EP,$_);
$pid = fork();
  if ($pid == 0) {
    print "\nI just opened $_\n";
      while (<EP>) {
        chomp;
        print "Contents of this file: $_\n";
      }
  } else {
    print "This is the parent process\n";
  }
}
print "\n";

[download]

Replies are listed 'Best First'.
Re: multiple fork()
by Dominus (Parson) on Apr 16, 2001 at 20:40 UTC
    The Async module may be helpful here. It takes care of the forking for you automatically; you just give it a subroutine to execute and it automatically executes the subroutine in a separate process.

    You get back an object which supports methods that allow you to ask if the subprocess has finished, and if so what the subroutine's return value was.

    If you're interested in how to handle the fork stuff yourself, the code for Async might be worth looking at; it's pretty simple.

    Happy Bicycle Day!

[reply]
      Don't I have to recompile my perl binary to support threading in order to use Async? I suppose this is doable, but then it means my code is even less portable than it already is using the telnet module.
[reply]
        The comment about a "separate process" makes me think that it is using fork. (Which is certainly how I would write it.) So you should not need to recompile, but you may have some problems on Windows. (Try it.)

        UPDATE
        Follow Dominus' link above for the Perl module under discussion...

[reply]
        Says Galen:
        Don't I have to recompile my perl binary to support threading in order to use Async?
        No; Async uses fork, which I believe is what you were asking for.

        Happy Bicycle Day!

        --
        Mark Dominus
        Perl Paraphernalia

[reply]
Re: multiple fork()
by spaz (Pilgrim) on Apr 16, 2001 at 21:10 UTC
    I would suggest you visit this node where I asked the same question and got some great answers.
    Parallel::ForkManager would probably serve you well.

    -- Dave
[reply]
Re: multiple fork()
by Rhandom (Curate) on Apr 16, 2001 at 22:37 UTC
    If you want something simple and light, here is some code that should take care of you 
    #!/usr/bin/perl


use IO::Select;
use IO::Pipe;

my $select = IO::Select->new();

my @commands = ("echo 1",
                "echo 2",
                "echo 3",
                "echo 4",
                "echo 5",
                );

foreach ( @commands ){

  my $pipe = IO::Pipe->new();

  my $pid = fork;
  die "Bad Fork $!" unless defined $pid;

  ### parent
  if( $pid ){
    $pipe->reader();
    $select->add( $pipe );

  ### child
  }else{
    $pipe->writer();
    $pipe->autoflush(1);

    print $pipe "PID $$: COMMAND $_\n";
    print $pipe `$_`; # do the command
    exit;
  }

}

my $num = 0;
while( my @responses = $select->can_read(0) ){
  die "Timedout [$!]\n" unless @responses;

  my $pipe = $responses[ rand @responses ];
  print STDOUT while <$pipe>;
  print "------------------------------------------------\n";

  $select->remove( $pipe->fileno() );

  last if ++$num >= @commands;
}

    [download]
    Using pipes to do all of the dirty work of IPC is great. Whoever made IO::Pipe and IO::Select (Graham Barr i think) made life wonderful. change @commands to whatever you want and you are done.
[reply]
[d/l]
      Oh - yeah, I didn't do anything with catching $SIG{CHLD} - There is the potential that you'll have zombie children until the parent process is done, but, once its done, they are gone too so I would worry about them too much unless the parent is a long, long running daemon.
[reply]

        Don't catch $SIG{CHLD} as each catch gives your program about a 1% chance of die'ing since Perl's signal handlers aren't safe. See "perldoc -f waitpid" for how to reap children without catching signals.

                - tye (but my friends call me "Tye")
[reply]
Re: multiple fork()
by ftforger (Sexton) on Apr 16, 2001 at 21:49 UTC
    Try doing your file open inside of the forked process.
[reply]
Re: multiple fork()
by traveler (Parson) on Apr 17, 2001 at 00:04 UTC
    Async looks like a useful module, however: please be aware that if you use AsyncTimeout it uses alarm which is incompatible with modules such as Gtk that use timeouts. This can cause a problem if your program has a Gtk GUI!

    --traveler

[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom