in reply to Converting a parallel-serial shell script

A solution using Proc::Queue:

One process is created to perform the data load while several (4) processes handle the data conversion. The conversors tell to the loader when they finish through a pipe.

Proc::Queue is used to ensure that only 4 conversors + the loader run at the same time

# untested: use Proc::Queue size => 4, qw(run_back); my $lpid = open my $loader, '|-'; defined $lpid or die "unable to fork: $!"; if ($lpid) { select $loader; $| = 1; select STDOUT; for (@ARGV) { run_back { if (system("convert.sh $payload") == 0) { print $loader "$payload\n"; } else { warn "Couldn't launch: $!/$?"; } } or warn "unable to fork: $!/$?"; } } else { while (<>) { system "load_db.sh $_") == 0 or warn "Couldn't launch: $!/$?"; } } 1 while wait != -1;