pepik_knize has asked for the wisdom of the Perl Monks concerning the following question:

Howdy monks,

I have a script that needs to look up prices on our intranet for about 40 items. Each inquiry takes about 15-20 secs, so I'd like to fork off several children to minimize the execution time. (I'm using Linux now, but plan on using perl2exe to place it on some win98 boxes.) Using the example here: how to fork?, I've got fork working, but how do I get the data back to the parent? The data that needs to get back are: price, description, and department. I know I can write the output to a file, and have the parent read out of that, but is there a better way?

Also, I'm unclear on the $SIG{CHLD} = sub { wait; };. Is this how I can make sure that my parent waits until the children are done? It doesn't seem to be doing that as written.

forktest.pl

#!/usr/bin/perl -w
use strict;
use LWP::Simple qw(get);
my $loops = shift || 10;
my $pid;
open (OUT, "+> out");
print "Testing with $loops children...\n";
for my $i (0..$loops-1) {
  if ($pid = fork) { ## Parent 
    $SIG{CHLD} = sub { wait; };
    next;
  }
  if (defined $pid) { ## ...children 
    my ($price,$desc,$dept);
    # So everyone can play along at home:
    my $url ="http://www.perlmonks.org/index.pl?node_id=".(6000+$i);
    my $page = get ($url);
    # extract data to $price,$desc,$dept
    # Now what?
    print OUT "$i $price $desc $dept\n";
    print "$i done\n";
    exit;
  }
  else { 
    die "Fork failed at number $i: $!\n";
  }
}

while <OUT> {
  # get data here?
}
print "I'm done\n";

[download]
Many Thanks,

Pepik

Of all the causes that conspire to blind
Man's erring judgment, and misguide the mind,
What the weak head with strongest bias rules,
Is pride, the never-failing vice of fools.
-- Pope.
(Humble to be American.)

Replies are listed 'Best First'.
Re: How do I get the child's data?
by Abigail-II (Bishop) on Jun 14, 2002 at 15:23 UTC
    One way of doing so is using pipes. Before you fork() open a pipe. After the fork(), the child writes, the parent reads. Luckely perl will all wrap that for you if you use open with a -| argument.

    The following example forks off 41 children using pipes. The children sleep for some time, then write data (this should be similar to your children). After having spawned off all the children, the parent is going to read all the pipes. It could very well be that the children are done in a different order than the parent is going to inspect them, but since you have to wait till the last one is finished, it doesn't matter (well, it would if there would be huge amounts of data involved). After reading all the data, we reap the children using wait, although that shouldn't be necessary since we didn't install a signal handler. 

    #!/usr/bin/perl

use strict;
use warnings 'all';
 
my @kids;
 
foreach my $count (0 .. 40) {
    my $pid = open $kids [$count] => "-|";
    die "Failed to fork: $!" unless defined $pid;
    unless ($pid) {
        # Child.
        sleep rand 60;  # Sleep for some random time.
        print "$$: ", int rand 0x7FFFFFFF, "\n";
        exit;
    }
}

foreach my $fh (@kids) {
    my @lines = <$fh>;
    print "Got: @lines";
}

1 while -1 != wait;

    [download]

    Abigail

[reply]
[d/l]
[select]
      Thanks, this looks like what I ought to do. I don't really understand
      print "$$: ", int rand 0x7FFFFFFF, "\n";
      Could someone explain that bit?
[reply]
[d/l]
        print "$$: ", int rand 0x7FFFFFFF, "\n";

        ...Is just some random data to show that your child process is doing something.
        $$ is the process ID of a process (in this case the child process).

[reply]
[d/l]
[select]
•Re: How do I get the child's data?
by merlyn (Sage) on Jun 14, 2002 at 15:20 UTC
[reply]
      You always say that! Thanks for the reference,

      Pepik

[reply]

Back to Seekers of Perl Wisdom