Many children, but never more than a fixed number at once.

Some times you have a need to fork of several children, but you want to limit the maximum number of children that are alive at one time. Here are two little subroutines that might help you, mfork and afork. They are very similar. They take three arguments, and differ in the first argument. For mfork, the first argument is a number, indicating how many children should be forked. For afork, the first argument is an array - a child will be forked for each array element. The second argument indicates the maximum number of children that may be alive at one time. The third argument is a code reference; this is the code that will be executed by the child. One argument will be given to this code fragment; for mfork it will be an increasing number, starting at one. Each next child gets the next number. For afork, the array element is passed.

Note that this code will assume no other children will be spawned, and that $SIG {CHLD} hasn't been set to IGNORE

sub mfork ($$&) {
    my ($count, $max, $code) = @_;

    foreach my $c (1 .. $count) {
        wait unless $c <= $max;
        die "Fork failed: $!\n" unless defined (my $pid = fork);
        exit $code -> ($c) unless $pid;
    }

    1 until -1 == wait;
}

sub afork (\@$&) {
    my ($data, $max, $code) = @_;
    my $c = 0;
    foreach my $data (@$data) {
        wait unless ++ $c <= $max;
        die "Fork failed: $!\n" unless defined (my $pid = fork);
        exit $code -> ($data) unless $pid;
    }

    1 until -1 == wait;
}
[download]

Comment on Many children, but never more than a fixed number at once. Select or Download Code

Replies are listed 'Best First'.
Re: Many children, but never more than a fixed number at once. by gav^ (Curate) on Jun 20, 2002 at 03:05 UTC
I find Parallel::ForkManager good for this kind of thing as it lets you write: `use Parallel::ForkManager; $pm = new Parallel::ForkManager($MAX_PROCESSES); foreach $data (@all_data) { # Forks and returns the pid for the child: my $pid = $pm->start and next; ... do some work with $data in the child process ... $pm->finish; # Terminates the child process }` [download] Code stolen from docs :) gav^	[reply] [d/l]
Re: Many children, but never more than a fixed number at once. by Abigail-II (Bishop) on Jul 08, 2002 at 16:33 UTC
Yeah, but that requires you to do three lines of code - to create an object (why? What's so OO about this?), and two calls (to `->start` and `->finish`) for each piece of code you want to execute. With `mfork` or `afork`, all you have is one call. But this is Perl, so there are more ways to do things. Abigail	[reply] [d/l] [select]