stephan_a has asked for the wisdom of the Perl Monks concerning the following question:

Dear Perl Monks,

my Perl program starts some threads that execute an external tool that I have to collect output from (STDOUT and STDERR). I'm using open3 from the IPC::Open3 package to start the child process and connect to its STDOUT and STDERR. Due to the tool manufacturer's somewhat weird idea of what an API is I have to execute that tool some hundred times per run.

Most of the time everything is fine but in some cases the open3 call blocks: It neither throws an exception nor returns. Yes, it's the open3 call itself and not the usual blocking problems with this function (I have solved them using select, sysread, and friends).

This is how I have implemented the call:

my $pid;
eval {
    $pid = open3(undef, \*CHLDOUT, \*CHLDERR, @$cmd);
};
if ($@) {
    ...
}

[download]

My Perl 5.8.9 (native 64 bit compilation) program runs on Solaris 10.

Do you have an idea what can make open3 hang/block and on how to continue debugging?

Thanks for your help and best regards,

Stephan

Replies are listed 'Best First'.
Re: open3 hangs
by salva (Canon) on Feb 19, 2009 at 09:35 UTC
    IPC::Open3 in order to work in as many platforms as possible has become so complex that its inners are almost inaccessible...

    Anyway, you have several options:

    • Use truss to see where (and why) the script is blocking at the OS level.
    • Make your own version of open3. For Unix it should be quite simple (see below).
    • ... or use some other CPAN module providing the same functionality (IPC::Run, IPC::Cmd, etc.). 
    # untested

use POSIX ();
sub my_open3 {
  pipe my($in_c), my($in_p) and
  pipe my($out_p), my($out_c) and
  pipe my($err_p), my($err_c) or croak "unable to create pipes";
  my $pid = fork;
  if (!$pid) {
    defined $pid or croak "fork failed";
    unless (open STDIN, '<&', $in_c and
            open STDOUT, '>&, $out_c and
            open STDERR, '>&, $err_c) {
      warn "redirections failed";
      POSIX::_exit(1);
    do { exec @_ };
    POSIX::_exit(1);
  }
  close $in_c;
  close $out_c;
  close $err_c;
  return ($in_p, $out_p, $err_p, $pid);
}

    [download]
[reply]
[d/l]
[select]

      Hi

      and thanks for your proposal. I have modified my code to use the old-fashioned pipe/fork/exec procedure showing me that open3 actually is not my problem. Instead fork blocks.

      I did some more investigation and found that forking from a thread can be tricky because one may have to deal with locking issues (fork seems to clone all threads). Now I'm looking for a way to cleanly fork from a (Perl) thread and exec afterwards. Any ideas?

      Kind regards,

      Stephan

[reply]
[d/l]
[select]
        Instead of forking, try creating a new thread and launching the child via system... maybe your libc system() call will use a different aproach that doesn't need to lock other threads, for instance, a vfork based one.
[reply]
[d/l]
[select]
Re: open3 hangs
by zentara (Cardinal) on Feb 19, 2009 at 12:50 UTC
    Just a few wild guesses....

    You have STDIN as undefined, I don't know if IPC3 likes that, it may want a 0 (true or false); and it may have to be a dummy filehandle in there to keep STDIN open for some reason. 

    my $pid = open3(  \*WRITE,  \*READ, 0,  "bc");
     #if \*ERROR is false, STDERR is sent to STDOUT

    [download]
    you may want to look at the examples in "perldoc -q stderr" and see how they use gensym and IO modules.

    If you don't need STDIN to the spawned program, why even use IPC::Open3? A plain piped open with 2>&1 should work, unless you absolutely need to separate STDOUT from STDERR.

    I'm not really a human, but I play one on earth My Petition to the Great Cosmic Conciousness
[reply]
[d/l]
Re: open3 hangs
by repellent (Priest) on Feb 20, 2009 at 00:04 UTC
    Maybe try:
    $pid = open3(\*CHLDIN, \*CHLDOUT, \*CHLDERR, @$cmd);
close(CHLDIN);

    [download]

    By the way, your usage of eval { .. }; if ($@) { .. } can lead to problems. Consider the eval { ..; 1 } or do { .. } idiom from this example:
    # timeout handling
{
    local $@ = "";
                                                                   
    eval {
        local $SIG{ALRM} = sub { die("alarm\n") }; # "\n" required
        alarm(3);

        # potentially long operation
        chomp($input = <STDIN>);

        alarm(0);
        1;
    } or do {
        die($@) unless $@ eq "alarm\n";
                                                                   
        # timed out
        warn("No answer for three seconds.\n");
        $input = "";
    }
}

    [download]
[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom