daverave has asked for the wisdom of the Perl Monks concerning the following question:

Hi, I use open( my $out, "-|", "some command" ); in order to print "some command" output on the fly.

I don't fully understand is the behavior of such open. Obviously, open is not blocking so the program continues. while the command output is continued being read. If I add

while (<$out>) { print; }
I actually wait for the command to finsih, right?

But if I don't, is there a way to check further down my program whether the command has finished (and if not, wait for it to finish)? Thanks, Dave

Replies are listed 'Best First'.
Re: open with pipe
by Corion (Patriarch) on Jul 31, 2010 at 13:19 UTC

    See the return value of open. I think on some operating systems, you can check with kill whether a process still exists under that pid:

    my $pid = open ...; kill 0 => $pid;

    Also see perlport, as mentioned in kill.

      Following your recommended reading, I now see that "closing any piped filehandle causes the parent process to wait for the child to finish" which solves my problem.

      Thanks!

Re: open with pipe
by ikegami (Patriarch) on Jul 31, 2010 at 19:03 UTC

    That will read until the pipe is closed, which is usually when the program ends, but it could be earlier.

    close($out) (done implicitly when $out goes out of scope) will wait for the program to end. It won't cause the program to end, though.

Re: open with pipe
by repellent (Priest) on Jul 31, 2010 at 21:28 UTC
    Given:
    my $child_pid = open( my $out, "-|", "some command" );

    The parent process opens a read filehandle $out from the child/command's STDOUT. This statement is non-blocking, as you observed. Note that if $child_pid is undefined, the command execution failed.

    If the child writes a lot to STDOUT, the child will eventually block when the pipe between the parent & child is full. It is up to the parent process to read data off from the pipe (read $out) so that the child may unblock to write more.

    However, merely reading from $out does not guarantee that the parent will "wait" for the command to finish. The child process could still be doing something else after it's last write to STDOUT, or even closing its STDOUT prior to doing more.

    To really wait for the command to finish:
    waitpid($child_pid, 0);
      Use close (implicitly or explicitly) with open, not waitpid.
      $ perl -E' open(my $pipe, "-|", "perl -e'\''sleep 5; exit 3'\''"); say time; close($pipe); say time; say $?>>8; ' 1280645564 1280645569 3
        Sure, close can wait for "-|". But why not waitpid?
        $ perl -E' $pid = open(my $pipe, "-|", "perl -e'\''sleep 5; exit 3'\''"); say time; waitpid($pid, 0); say time; say $?>>8; ' 1280685258 1280685263 3

        waitpid is documented with:

          Waits for a particular child process to terminate and returns the pid of the deceased process, or -1 if there is no such child process.

          The status is returned in $? and ${^CHILD_ERROR_NATIVE}.

Re: open with pipe
by dasgar (Priest) on Aug 01, 2010 at 05:37 UTC

    If I read your post correctly, you're basically wanting to issue a system command from Perl and read the output of that command back into Perl. Based on that, I personally would go with what I consider in my mind to be the "quick and dirty" method, which is to use back ticks to issue the command.

    In my sample code below, I'm storing a command in $cmd and then using back ticks issue that command and store the output in $cmd_out.

    use strict; my $cmd = 'cat /proc/cpuinfo'; my $cmd_out = `$cmd`;

    There's a few things to keep in mind with this method:

    • Your Perl script will wait until the system command is complete before continuing.
    • This only gives you STDOUT. If there's an error message sent to STDERR, that output is not captured.
    • I believe that the issued command only has the same level rights as the user account used to run the Perl script.
    I'm not advocating that this is the best route to go, but it usually gets the job done for this lazy programmer. :D

      Simple and innocent looking backticks become a nightmare once you write code for more than one system. Some problems, some general, some with your example:

      • You have to work with an unknown shell. Many shells behave according to POSIX, others don't. You don't know what shell you get. You can't write reliable code this way.
      • You don't know the quoting rules of the shell, because you don't know the shell. You can't reliably quote program name and program arguments. You don't even know WHEN to quote names and arguments. On Windows, things are even worse. You may end passing invalid arguments to the invoked program.
      • You don't know the I/O redirection rules of the shell, because you don't know the shell. < and > are pretty universal, but 2> isn't supported everywhere (e.g. command.com doesn't know that). Don't even think about 2>&1.
      • /proc/cpuinfo is Linux specific. For example, FreeBSD doesn't have /proc/cpuinfo. Don't even think about Windows.
      • cat is called without a path, so you will execute the first cat program found in $ENV{'PATH'}. Maybe you execute /usr/bin/cat, the program that concatenates files and prints them to STDOUT. Maybe you execute /home/evil/hoho/cat, a program that changes the root password and spawns a telnet daemon on port 37331 before pretending to be the innocent /usr/bin/cat. No big problem as long as you invoke that cat as unprivileged user, but when run as root, you loose.
      • There is no cat on a fresh Windows system. A similar command is called type and is a shell built-in. Remember that it has a different behaviour (it adds output lines, both empty and with filenames, when called with more than one argument).
      • There is no need to use cat inside backticks to read a file into a scalar: open my $h,'<','/proc/cpuinfo' or die "Could not read /proc/cpuinfo: $!";my $result=do { local $/=<$h> }; close $h;.
      • Neither is there a need to use cat inside backticks to read a file into an array: open my $h,'<','/proc/cpuinfo' or die "Could not read /proc/cpuinfo: $!";my @result=<$h>; close $h;.

      Alexander

      --
      Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

        You have to work with an unknown shell. Many shells behave according to POSIX, others don't. You don't know what shell you get. You can't write reliable code this way.

        It's sh except on Windows, and that's easy to check.

        @afoken,

        Thanks for expanding the list of reasons why this is not the best idea of what to do for all cases. I knew there were probably more reasons than I could think of as to why this method could lead to trouble. As I mentioned, I do consider this a "quick and dirty" method because it can lead to lots of problems if you're not careful.

        So far, I've been too lazy to try to figure out how to properly use the IPC::Open3 with proper error trapping. I tried unsuccessfully a few times and then gave up and used back ticks instead. I've also been lucky enough so far to not have been bitten in the rear when I've used back ticks. :D

        Also, I could be wrong, but I believe that some of your cautionary warnings about OS specific commands and syntax would apply to any method in any programming language when trying to call system commands.

        And yes, I probably put up a poor example. Considering it was about 12:30 am in my time zone when I posted, I should have waited until I was thinking more straight.