Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

How the parent process knows that all its child process has finished execution?

by kishore036 (Novice)
on May 24, 2008 at 12:33 UTC ( [id://688282]=perlquestion: print w/replies, xml ) Need Help??

kishore036 has asked for the wisdom of the Perl Monks concerning the following question:

Hi, The parent process fork's and creates 30-40 child process to process 30-40 files. The requirement is that the parent process has to wait until all child process have completed processing the files. How will the parent process know that all child process has finished execution? Thanks, Kishore

Replies are listed 'Best First'.
Re: How the parent process knows that all its child process has finished execution?
by moritz (Cardinal) on May 24, 2008 at 12:46 UTC
Re: How the parent process knows that all its child process has finished execution?
by almut (Canon) on May 24, 2008 at 12:46 UTC

    The parent process could store all PIDs of its children, and every time a child terminates, it would remove the respective PID from that set in the SIGCHLD handler (waitpid returns the PID of the deceased process). When the set is empty, all immediately forked processes are gone.

Re: How the parent process knows that all its child process has finished execution?
by zentara (Archbishop) on May 24, 2008 at 17:17 UTC
Re: How the parent process knows that all its child process has finished execution?
by Anonymous Monk on May 24, 2008 at 12:44 UTC
Re: How the parent process knows that all its child process has finished execution?
by casiano (Pilgrim) on May 25, 2008 at 11:43 UTC
    The following example can help you.
    The main process forks a number of children with logical IDs between 0 and $np. The processes concurrently gain exclusive access to a file using a lock.
    lhp@nereida:~/Lperl/src/perl_networking/ch2$ cat -n flock.pl 1 #!/usr/bin/perl -w 2 use strict; 3 use Fcntl qw(:DEFAULT :flock); 4 use POSIX qw(WNOHANG); 5 $| = 1; 6 7 local $SIG{CHLD} = sub { 8 while (my $kid = waitpid(-1, WNOHANG) > 0) {} 9 }; 10 11 sub create_child { 12 my ($id, $task) = splice @_, 0, 2; 13 my $pid; 14 15 return $pid if $pid = fork(); 16 die "Cannot fork $!" unless defined $pid; 17 $task->($id, @_); # do something 18 exit; 19 } 20 21 sub parfor { 22 my $LAST = shift; 23 my $task = shift; 24 my @pid; 25 26 $pid[0] = $$; 27 $pid[$_] = create_child($_, $task, @_) for 1..$LAST; 28 return @pid; 29 } 30 31 sub task { 32 my $id = shift; 33 my $fn = shift; 34 35 sleep(int(rand(2))); 36 open my $f, "+<$fn" or die "Can't open $fn. $!\n"; 37 flock($f, 2); 38 seek $f, 0, 0; 39 my $num = <$f>; 40 warn "Process $id reads $num.\n"; 41 seek $f, 0, 0; 42 my $s = $num+$id; 43 warn "Process $id writing $s.\n"; 44 print $f $s; 45 close($f); 46 exit; 47 } 48 49 #main 50 my $np = shift || 3; 51 my $fn = shift || "sync.txt"; 52 open my $f, "> $fn" or die "Can't open file $fn. $!\n"; 53 print $f 0; 54 close($f); 55 &parfor($np, \&task, $fn); 56 do {} while wait > 0;
    The answer to your question is in line 56

    Hope it helps

    Casiano

Re: How the parent process knows that all its child process has finished execution?
by jdrago_999 (Hermit) on May 26, 2008 at 01:40 UTC
    If you can install the forks module, your code could be very simple:
    use forks; my $max_threads = 30; for( 1...$max_threads ) { threads->create( \&munge_files, shift(@files_to_munge) ); }# end for() # Wait for our threads to finish: $_->join foreach threads->list; # All done! print "Job was completed.\n"; sub munge_files { my ($file) = @_; # Do stuff with $file... }
Re: How the parent process knows that all its child process has finished execution?
by absolut.todd (Monk) on May 27, 2008 at 07:00 UTC
    I used Randal Schwartz's forking parallel link checker as the base for a application that needed to run many children in parallel. It solves all your problems, even down to managing it so you can configure the number of child processes running at once.

    It works really well and i have been happy with it! It cut the processing time in the application from 45 minutes to about 15!!

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://688282]
Approved by moritz
Front-paged by tye&nbsp;
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others admiring the Monastery: (5)
As of 2024-03-28 17:09 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found