As far as I can see, all your processes funnel into the same pipe. This means that their output will be intermingled, and I mean really intermingled, not just on a line by line basis. So if one program says "aaa\nbbb\n" and the other says "ccc\nddd\n", in principle you may get "aaca\nbcbb\nc\nddd\n".

Depending on the OS, often only one message can be in transit on a pipe at any one time and if the output is for example linebuffered and will fit totally in the pipe you often don't notice this possibility.

But if output blobs can get big, or are blockbuffered, some other messages might succeed in mixing in. I suspect that this is what is happening here, and if you study your arrays carefully enough, you will find the missing hits as strings in the @line array (or not at all if they got added to an Error).

Several solutions are possible. E.g. if you control the target programs tightly enough and all lines are shorter than 512 bytes, you can make sure to always flush them. Or you can set up multiple pipes and collect from them using select() or poll(), though you'll have to collect lines yourself in that case (things like POE can help there). Or you can redirect all output to a file per program, and process these files when the programs are done.

PS, notice that you should also check for the defined-ness of $pid to make sure the implied fork worked. Also, your child pipe setup is overly complicated since you don't really use these pipes (currently). You can use a fork/exec there.

update.

Here is an example of a multi-pipe select based solution:

#!/usr/bin/perl -w
use strict;
use IO::Select;

use constant READ_SIZE => 8192;

my @program = qw(cat /etc/passwd);
my $select = IO::Select->new();
my %collect_line;

for (1..4) {
    open(my $fh, "-|", @program) || die "Could not start @program: $!"
+;
    $select->add($fh);
    $collect_line{$fh} = "";
}

sub line {
    my $line = shift;
    # Do your per line processing here
    print "Process $line";
}

while (%collect_line) {
    for my $fh ($select->can_read) {
        my $rc = sysread($fh, $collect_line{$fh}, 
                         READ_SIZE, length($collect_line{$fh}));
        die "Read error $! from pipe ??" if !defined($rc);
        if ($rc) {
            line($1) while $collect_line{$fh} =~ s/^(.*\n)//;
        } else {
            # EOF
            line($collect_line{$fh}) if $collect_line{$fh} ne "";
            delete $collect_line{$fh};
            $select->remove($fh);
            close($fh);
            die "Unexpected $? returncode from @program\n" if $?;
        }
    }
}
[download]

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.