in reply to Why is a deeply-bound lexical in a forked closure blocking the parent process?

This may be full of it, it's a long time since I messed around with this in perl, and Back When I wasn't using pipes this way - I was using files.

The parent blocks because $psyscmd goes out of scope upon the return in the parent process, and perl tries to destroy the filehandle (because the only living reference to it is from the structure referenced by $psyscmd) but finds its close() blocking on the child.

In the global case, the parent does not try to close() the file so it doesn't show up, but in the lexical case it does, because perl wants all open files to close as soon as they go out of scope (in case you rely on the behavior and try to open them again).

The file is (after fork()) held open by two processes, and perl doesn't want to lie to you about the success of close(), nor fail just because someone else is looking at the file, so (apparently) it blocks.

If you return a reference to $psyscmd it shouldn't block, because $psyscmd doesn't then have to be destroyed, and you can sync up at a later point (like when leaving the program).

I don't know that you can open a pipe handle with shared semantics, but that might be another way to close your handle without blocking, if in fact you can and perl is smart enough to know that you did (I'm sure perl is smart, it's whether you can share your end of the pipe).

Or maybe I'm just dead wrong and wasting valuable electrons... I'll be interested to see what others think.

You should be able to test my hypothesis by issuing an intentional close() on the filehandle in the parent before you return (and putting in another print). If it blocks there, then I'm at least a little bit right.

  • Comment on Re: Why is a deeply-bound lexical in a forked closure blocking the parent process?

Replies are listed 'Best First'.
Re^2: Why is a deeply-bound lexical in a forked closure blocking the parent process?
by liverpole (Monsignor) on Feb 24, 2006 at 14:20 UTC
    Your answer is not only at least a little bit right, it's exactly on target!  Excellent analysis! ++

    I read your response yesterday, and was fairly sure that you had hit the nail on the head.  But I wanted to do it justice by testing further, and as I expected, the program succeeds when another reference is created to the filehandle.

    First, I made sure $b_use_global_fh was zero, and changed system_command() to return both $psyscmd and the filehandle $fh, so I could create a reference to $fh to keep it open: 

        my ($psyscmd, $fh) = system_command("$bld_cmd 2>&1");
    $global_fh = $fh;

    [download]
    That worked exactly as you predicted.  It also worked when I simply had the parent process return the filehandle: 
        if ($pid) {
        # Parent
        print STDERR "\e[101m(1a) Parent about to return\e[m\n";
        return $fh;
    }

    [download]
    but only, of course, when the reference was preserved after the return from monitor_build: 
    my $unused_fh = monitor_build("./fakebuild");

    [download]

    So I thank you for a very enlightening solution.  It taught me something important about closures; namely, that they close their open filehandles which go out of scope, just the way a program does when it exits.  Nicely written!

    Update:  I just realized (and just tested as correct) that another way to keep the filehandle from attempting to close is for the parent to return the reference to the closure: 

        my $psyscmd = system_command("$bld_cmd 2>&1");
    
    # ...

    if ($pid) {
        # Parent
        print STDERR "\e[101m(1a) Parent about to return\e[m\n";
        return $psyscmd;
    }

    [download]
    Then, as long as it is saved in the main program, this technique works as well.  It also avoids having to fuss with the filehandle directly.
    @ARGV=split//,"/:L"; map{print substr crypt($_,ord pop),2,3}qw"PerlyouC READPIPE provides"
[reply]
[d/l]
[select]
      Hi. This is my first time on perlmonks - I am a co-worker of liverpole's, and he and I have been discussing his question. I hope I am not out-of-line by continuing the discussion.

      (I was close-but-off when we discussed it here at work. I'd surmised it was I/O blocking of some type, but had presumed it was related to code for output that he's since eliminated. It wasn't the problem.)

      I hadn't considered file closures - because to my limited understanding of UNIX file systems (liverpole is using one of the Red Hat Linux systems), file closure was relatively independent from one system process to another. A process on UNIX will close a file, any unwritten buffers will write out, any reference counts will be decremented, and the process will continue. (The kernel may take further action if reference counts reach zero, but that is outside of the process, and irrelevant to it.)

      Clearly, you have demonstrated that there is a coordination of file closures, and that the main parent was blocking on closure at an inconvenient time (at the end of the local scope). It doesn't surprise me, therefore, that moving the file handle or keeping a reference count to the file handle until the program exits will postpone that blocking until it no longer matters. (Other than efficiency, who cares if the parent program blocks during exit, until the child also exits?)

      But - what in the world is causing/permitting the coordination between the two processes? Is it a Perl construct to do so? What if the child, instead of simply forking, performed an exec as well - would that prevent or reduce any coordination?

      I'm truly convinced that there is unexpected coordination of the files on file closure (whether that file closure is implicit or explicit). But I'm not a Frater of Perl enough to know how or why such coordination is taking place?

      Help? Don't worry, liverpole will tell me if anyone replies. Thanks for reading, and tolerating. And answering.

[reply]

In Section Seekers of Perl Wisdom