Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Can't close pipe to invalid process

by cLive ;-) (Prior)
on Aug 28, 2015 at 21:37 UTC ( #1140381=perlquestion: print w/replies, xml ) Need Help??

cLive ;-) has asked for the wisdom of the Perl Monks concerning the following question:

I was debugging why this test to create a PDF worked in the shell, but not from Jenkins (both environments running as the same user):
my $fh; ok( open($fh, "|-", "prince - 2>/dev/null $test_file"), "Open +$test_file for writing" ); print $fh "<html><body><h1>Hello, world!</h1></body></html>"; ok( close($fh), "Close $test_file");

Turns out it was because the PATH didn't exist in the Jenkins environment.

Problem solved, but now I'm left scratching my head.

ok 3 - Open /opt/humana/svn/checkouts/rosalind-repo/prod/tmp/testfile. +pdf for writing not ok 4 - Close /opt/humana/svn/checkouts/rosalind-repo/prod/tmp/test +file.pdf

Why would the open succeed to an unknown executable? Is it because it dosen't fail until you attempt to use it? For some reason that feels a little off to me. What am I missing?

Replies are listed 'Best First'.
Re: Can't close pipe to invalid process
by aitap (Curate) on Aug 29, 2015 at 16:05 UTC

    I think that this behaviour is caused by the fact that Perl has to launch a shell to process a command given in a string form, and the open doesn't fail because shell does launch successfully (it's the command that fails later):

    use Test::Simple tests => 3; ok !open(my $fh, '|-', 'nonexistent program'), 'string form of open fa +ils'; ok open(my $prince, '|-', 'nonexistent program 2>/dev/null'), 'but suc +ceeds if shell uses redirection'; ok !open(my $none, '|-', 'nonexistent', 'program'), 'list form of open + fails, as it should'; __END__ 1..3 ok 1 - string form of open fails ok 2 - but succeeds if shell uses redirection ok 3 - list form of open fails, as it should

    But why does string form of open correctly fail in the first test? Perl is able to process a simple command without help of a shell, as strace shows:

    $ strace -f -e clone,execve perl -e'open(my $fh, "|-", "nonexistent pr +ogram")' execve("/usr/bin/perl", ["perl", "-eopen(my $fh, \"|-\", \"nonexisten" +...], [/* 45 vars */]) = 0 clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIG +CHLD, child_tidptr=0xb74f7a68) = 7792 Process 7792 attached [pid 7792] execve("/home/aitap/perl5/bin/nonexistent", ["nonexistent" +, "program"], [/* 45 vars */]) = -1 ENOENT (No such file or directory +) [pid 7792] execve("/home/aitap/scripts/nonexistent", ["nonexistent", +"program"], [/* 45 vars */]) = -1 ENOENT (No such file or directory) [pid 7792] execve("/home/aitap/bin/nonexistent", ["nonexistent", "pro +gram"], [/* 45 vars */]) = -1 ENOENT (No such file or directory) [pid 7792] execve("/usr/local/bin/nonexistent", ["nonexistent", "prog +ram"], [/* 45 vars */]) = -1 ENOENT (No such file or directory) [pid 7792] execve("/usr/bin/nonexistent", ["nonexistent", "program"], + [/* 45 vars */]) = -1 ENOENT (No such file or directory) [pid 7792] execve("/bin/nonexistent", ["nonexistent", "program"], [/* + 45 vars */]) = -1 ENOENT (No such file or directory) [pid 7792] execve("/usr/local/games/nonexistent", ["nonexistent", "pr +ogram"], [/* 45 vars */]) = -1 ENOENT (No such file or directory) [pid 7792] execve("/usr/games/nonexistent", ["nonexistent", "program" +], [/* 45 vars */]) = -1 ENOENT (No such file or directory) [pid 7792] +++ exited with 1 +++ --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=7792, si_uid +=1000, si_status=1, si_utime=0, si_stime=0} --- +++ exited with 0 +++
    Stream redirection, however, requires a shell:
    $ strace -f -e clone,execve perl -e'open(my $fh, "|-", "nonexistent pr +ogram 2>/dev/null")' execve("/usr/bin/perl", ["perl", "-eopen(my $fh, \"|-\", \"nonexisten" +...], [/* 45 vars */]) = 0 clone(Process 7809 attached child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, +child_tidptr=0xb753ea68) = 7809 [pid 7809] execve("/bin/sh", ["sh", "-c", "nonexistent program 2>/dev +/null"], [/* 45 vars */]) = 0 [pid 7809] +++ exited with 127 +++ --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=7809, si_uid +=1000, si_status=127, si_utime=0, si_stime=0} --- +++ exited with 0 +++

    Edit: spelling

        Correct me if I'm wrong, but OP said:

        Turns out it was because the PATH didn't exist in the Jenkins environment.
        Why would the open succeed to an unknown executable?
        As far as I understood OP, missing PATH meant that the script was unable to run executables except by full path:
        $ env LC_ALL=C PATH="" /usr/bin/perl -E'open my $fh, "-|", $_ and say +"success" or say $! for "ls", "/bin/ls"' No such file or directory success
        So the question was: how does open succeed despite inability to find the executable due to empty $PATH?

        By the way, thanks for the spelling reminder.

        It was the not finding "prince" that caused it to fail (pretty much what aitap said).

        Intuitively, I would think the open should fail if the executable being piped to can't be found, but that wasn't the case.

        I added a test for the print, and that appears to return true.

        So, what I have is:

        ok 3 - Opened pipe ok 4 - Piped content to prince not ok 5 - Closed pipe

        I'm wondering if an assumption is made internally about piping to the executable until Perl receives a return value on closing?!?

Re: Can't close pipe to invalid process
by Anonymous Monk on Aug 28, 2015 at 22:03 UTC

    What am I missing?

    Maybe a simpler test case with more diagnostics ( %!, $? )

    It could die after the open

    Also it could be possible for print to fail and/or kill this "prince" process

      Well, the print wasn't dying, because the close() test actually runs. BTW, prince is an archaic PDF generator that I've inherited, so I'm just adding tests as I go.

      I guess my concern is why the invalid open returned true. Seems counterintuitive.

        I guess my concern is why the invalid open returned true.

        Because the (piped) open succeeded. It found and successfully ran the prince executable.

        The problem is that you are attributing the failed close to the failure to find the non-existent file; which is wrong.

        What you are attempting to close is the pipe to the prince executable; which fails because the executable closed its end of the pipe, when it terminated because it couldn't find the file.

        Bottom line: You cannot close the pipe because it is already closed. The confusion arises because you are attributing the failed close to the failure of the file to exist; rather than the fact the the pipe was already closed.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
        In the absence of evidence, opinion is indistinguishable from prejudice.
        I'm with torvalds on this Agile (and TDD) debunked I told'em LLVM was the way to go. But did they listen!

        Well, the print wasn't dying, because the close() test actually runs.

        You mean in your updated new code? Because the code you posted doesn't die if print fails.

        I guess my concern is why the invalid open returned true. Seems counterintuitive.

        Um, if open succeeded, and print succeeded, then it must not have been an invalid open, so nothing counterintuitive there

        Its very possible for a program to end after a successful open to it

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1140381]
Approved by Old_Gray_Bear
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (4)
As of 2022-08-12 03:51 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?