in reply to Processes clobbering each other
Depending on the OS, often only one message can be in transit on a pipe at any one time and if the output is for example linebuffered and will fit totally in the pipe you often don't notice this possibility.
But if output blobs can get big, or are blockbuffered, some other messages might succeed in mixing in. I suspect that this is what is happening here, and if you study your arrays carefully enough, you will find the missing hits as strings in the @line array (or not at all if they got added to an Error).
Several solutions are possible. E.g. if you control the target programs tightly enough and all lines are shorter than 512 bytes, you can make sure to always flush them. Or you can set up multiple pipes and collect from them using select() or poll(), though you'll have to collect lines yourself in that case (things like POE can help there). Or you can redirect all output to a file per program, and process these files when the programs are done.
PS, notice that you should also check for the defined-ness of $pid to make sure the implied fork worked. Also, your child pipe setup is overly complicated since you don't really use these pipes (currently). You can use a fork/exec there.
update.
Here is an example of a multi-pipe select based solution:
#!/usr/bin/perl -w use strict; use IO::Select; use constant READ_SIZE => 8192; my @program = qw(cat /etc/passwd); my $select = IO::Select->new(); my %collect_line; for (1..4) { open(my $fh, "-|", @program) || die "Could not start @program: $!" +; $select->add($fh); $collect_line{$fh} = ""; } sub line { my $line = shift; # Do your per line processing here print "Process $line"; } while (%collect_line) { for my $fh ($select->can_read) { my $rc = sysread($fh, $collect_line{$fh}, READ_SIZE, length($collect_line{$fh})); die "Read error $! from pipe ??" if !defined($rc); if ($rc) { line($1) while $collect_line{$fh} =~ s/^(.*\n)//; } else { # EOF line($collect_line{$fh}) if $collect_line{$fh} ne ""; delete $collect_line{$fh}; $select->remove($fh); close($fh); die "Unexpected $? returncode from @program\n" if $?; } } }
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re^2: Processes clobbering each other (atoms)
by tye (Sage) on Nov 24, 2003 at 23:14 UTC | |
by thospel (Hermit) on Nov 24, 2003 at 23:23 UTC | |
by mcogan1966 (Monk) on Nov 25, 2003 at 13:54 UTC | |
|
Re: Re: Processes clobbering each other
by mcogan1966 (Monk) on Nov 25, 2003 at 15:08 UTC |