tange has asked for the wisdom of the Perl Monks concerning the following question:

open3 only takes STDIN, STDOUT and STDERR. If the command being run uses other file descriptors then open3 cannot capture those: 

    echo foo # Can be captured
    echo foo >&2 # Can be captured
    echo foo >&3 # Cannot be captured (with open3)

[download]
I have looked into IPC::Run::run which seems to be able to deal with that, but I also need the PID and I found no way for IPC::Run::run to give me that. Also IPC::Run is not installed on many of the old distributions supported by GNU Parallel, whereas open3 is.

How can I wrap open3 so that I can capture file descriptors 3..61 in the same way I can capture STDOUT and STDERR?

Background

This is intended for a possible extension of GNU Parallel, so you can do: 

    # Currently mixes output due to fd 3 not buffered
    parallel 'echo {} start >&3;sleep 10;echo {} end >&3' ::: a b c 3>
+out.file

[download]
without having the output from different jobs mixed. Right now only STDOUT and STDERR are buffered and thus guaranteed not to mix. 
    # This currently works
    parallel 'echo {} start >&2;sleep 10;echo {} end >&2' ::: a b c 2>
+out.file

[download]

Replies are listed 'Best First'.
Re: open3 but for n file descriptors
by Loops (Curate) on Jul 19, 2013 at 19:32 UTC

    Are you sure you really need to have the PID of child processes when using IPC::Run? It allows you to send signals and kill children directly. If you really must have the PID, it's possible to use start/pump/finish instead of run:

    use strict;
use warnings;
use IPC::Run qw( start );

# Initiate the child process
my $data;
my $s = start [ 'ls' ], '>', \$data;

# Show its PID
for my $kid ( @{$s->{KIDS}} ) {
    print $kid->{PID};
}

# pump data in and out, then clean up
$s->pump;
$s->finish;

    [download]
[reply]
[d/l]

      As mentioned I would prefer not to use IPC::Run because IPC::Run is not installed on many of the old distributions supported by GNU Parallel, whereas open3 is. There is typically a reason why people do not upgrade the old systems, and it is therefore unlikely that IPC::Run will be installed.

      The PID is currently the way I keep track of which process was started when, so when using --keep-order, I can make sure the output from the dead children is printed in the correct order. But with a bit of recoding this could probably be changed.

      I had hoped wrapping open3 with more filedescriptors would be as easy as this meta-code for fd3: 

      # Record fd3 for later use
open($orig_fd{3},">&=",3);
for $id (1..10) {
  open($child{$id,3},">",$tmpfile{$i});
  # make $child{$id,3} file descriptor 3 for me and all future childre
+n
  connect_filedescriptor(3,$child{$id,3});
  # start the child
  open3(...)
}

sub reaper {
  # When a child dies
  seek $child{$id,3},0,0;
  # Print the captured fd3 to the parent's original fd3
  print $orig_fd{3} <$child{$id,3}>;
  close $child{$id,3};
}

      [download]
      I know how to make most of the meta-code, but connect_filedescriptor I do not know how to make.

[reply]
[d/l]
        Looking at the man page for open, it seems what I need is this: 
        open(my $err, '>', "foo.out"); 
open(STDERR, ">&", $err);

        [download]
        but in a generic version, so I can use other file descriptors than STDERR (fd2).
[reply]
[d/l]

Back to Seekers of Perl Wisdom