Deadlock occurring, reason unknown

premchai21 has asked for the wisdom of the Perl Monks concerning the following question:

I am writing a program which will allow the user to create a system of pipes in a text file, which can then be interpreted and executed, with a bunch of processes, and one or more inputs and one output for each process, which may be linked to STDIN, STDOUT, another process, or nowhere. (I'm planning to include possibilities for more than one output for each process, capturing / outputting to STDERR, etc. but I need to fix this version before adding anything.) However, deadlock is occurring for no apparent reason. The relevant code:

$water = {
map
{
    my $process = $_;
    my @inputs = @ {$processes{$process}[1]};
    $_, {
    prerequisites => [ @inputs ],
    oo_output =>
        sub
        {
        my ($data) = shift;
        my ($pid, @fh, $active);
        @fh = map { $data->get($_) } @inputs;
                push @fh, \*STDIN if ((defined $start) and $start eq $
+process);
                $active = scalar @fh;
        local (*IN, *OUT, $/, $\);
        $/ = 1;
        $\ = '';
        open2 \*OUT, \*IN, $processes{$process}[0];
        select(IN);
        for (;;)
        {
            my $piece = join '', map { (eof $_) ? "\0" : <$_> } @fh;
            (grep { not eof $_ } @fh) ? (print $piece) : (last);
        }
        close (IN);
        return \*OUT;
            } } } keys %processes };

tie %river, 'Data::Flow', $water;
{
    my $output = $river{$final};
    local $/ = undef;
    local $_;
    $_ = <$output>;
    print;
}
exit 0;
[download]

... where the Data::Denter patterns of the variables are approximately: # \%processes % <name> => @ <command> @ <inputs' names, one per element> # \@pipes @ @ <from> <to> # $start <which process connects to STDIN> # $final <which process connects to STDOUT> ... and the data file: process 1stcat perl -spew ..\gismu.text process 2ndcat perl -spew ..\cmavo.text process 3rdcat perl -spew pipe 1stcat 3rdcat pipe 2ndcat 3rdcat final 3rdcat ... and a prompt transcript under Win32 (#s are special actions): C:\premchai21\perl>perl multipipe.pl -f pipes #hang # #kill.# -p destination: Broken pipe -p destination: Broken pipe #hang # #kill!# C:\premchai21\perl> I tried it in the Perl debugger too, but it was rather unhelpful. What is going wrong? Running ActiveState Perl 5.6.0, build 623, on Win98.

Comment on Deadlock occurring, reason unknown Download Code

Replies are listed 'Best First'.
Re (tilly) 1: Deadlock occurring, reason unknown by tilly (Archbishop) on Jul 08, 2001 at 18:59 UTC
My immediate guess is that someone, somewhere, is buffering input and/or output and that is leading to deadlock. But tracing your intended trace of actions, that isn't the case. My second guess is that this has something to do with the emulation on Windows of Unix functionality. I see a call to IPC::Open2, which is a wrapper around IPC::Open3. Under NT I have had mixed success. I have run stuff and had it work. But it failed with more complex stuff. Dunno if that is the problem, but there is absolutely no way that it would be a bad idea to test the success of your open2 and close calls. Plus when you close out a process you probably need to explicitly wait to pick up your children (and be able to trap error information from them). Now doing all that probably will not solve your problem directly, but there is a good chance that it will at the least give you a far more specific set of error messages to stare at...	[reply]
Re: Deadlock occurring, reason unknown by toma (Vicar) on Jul 08, 2001 at 23:19 UTC
I'm sure I don't understand your code, but one thing looks funny to me. That is, you select a filehandle and then close it. I think you need to select something else somewhere, because a filehandle remains selected even after it has been closed. In unix, this can sometimes go unnoticed because the filehandle for the next file that is opened will be the next one that is available, which may be the one that you last closed. Here is some code that shows the idea: `use strict; open(IN, "datafile") or die "can't open datafile"; local $/= 1; select(IN); $_= <IN>; close IN or die "can't close IN"; print "datafile:\n", $_ or die "can't print";` [download] This code dies with a `can't print` because `IN` is still selected. This example is repaired by adding a `select STDOUT` statement right before the `print`. You may want to add error checking to your code not just for your open commands, but also on your print and close statements. It should work perfectly the first time! - toma	[reply] [d/l] [select]
Re: Deadlock occurring, reason unknown by premchai21 (Curate) on Jul 09, 2001 at 05:24 UTC
As tilly and toma pointed out, error checks might be useful; to quote approximately from Camel, 2nd edition: """ Always check the return values of system calls. Always check the return values of system calls. ALWAYS CHECK THE RETURN VALUES OF SYSTEM CALLS! """ Which I of course forgot to do. Oops. Will add error checks and reply again if I still can't figure out the problem. Update: Well, that didn't seem to be the problem, because I added checks, and a wait (marked by ##): $water = { map { my $process = $_; my @inputs = @ {$processes{$process}[1]}; $_, { prerequisites => [ @inputs ], oo_output => sub { my ($data) = shift; my ($pid, @fh, $active); @fh = map { $data->get($_) } @inputs; push @fh, \STDIN if ((defined $start) and $start eq $ +process); $active = scalar @fh; local (IN, OUT, $/, $\); $/ = 1; $\ = ''; open2 \OUT, \IN, $processes{$process}[0] or die "Can't open2 \"${\($processes{$process}[0])}\": $!, $? +, $^E; stopped"; ## select(IN) or die "Can't select IN: $!, $?, $^E; stopped"; ## for (;;) { my $piece = join '', map { my $x; (eof $_) ? "\0" : (defin +ed ($x = <$_>) ? $x : ( die "Can't read: ", ## "$!, $?, $^E; stopped" )) } @fh; (grep { not eof $_ } @fh) ? (print($piece) or die "Can't print: $!, $?, $^E; stopped") # +# : (last); } close (IN) or die "Can't close IN: $!, $?, $^E; stopped"; ## return \OUT; } } } keys %processes }; tie %river, 'Data::Flow', $water; $SIG{CHLD} = sub { wait }; ## { my $output = $river{$final}; local $/ = undef; local $_; defined($_ = <$output>) or die "Can't read: $!, $?, $^E; stopped"; + ## print or die "Can't print: $!, $?, $^E; stopped"; ## } exit 0; [download] ... and it no longer hangs at first, but then the pipes break, spilling data everywhere, and it hangs: `C:\premchai21\perl>perl multipipe.pl -f pipes -p destination: Broken pipe -p destination: Broken pipe #kill.# #darn! killed emacs instead# #kill!# C:\premchai21\perl>` Tried it with a smaller set, "tiny", listed: `process cat perl -spew start cat final cat` Prompt transcript (read STDIN ok, ##s are what I type but don't see): `C:\premchai21\perl>perl multipipe.pl -f tiny hello, world! #^Z, RET# #^Z, RET# #^Z^Z^Z^Z^Z^Z^Z^Z^Z# at multipipe.pl line 102. C:\premchai21\perl>` Line 102 is the line with the defined test on the read from $output, third from the end. So something seems to be wrong with the read, but as the die didn't function properly, I still can't tell exactly what's wrong. Was hoping for a more informative error message, which I didn't get. Anyone?	[reply] [d/l]
Re: Re: Deadlock occurring, reason unknown by premchai21 (Curate) on Jul 12, 2001 at 05:56 UTC
Bleh. Forgot to reselect STDOUT before the final print. So, added `select(STDOUT);` right before the print line. Then: `C:\premchai21\perl>perl multipipe.pl -f tiny hello, world! #^Z, RET# #^Z, RET# #^Z^Z^Z^Z^Z^Z^Z^Z^Z# Can't read: Bad file descriptor, 36081, ; stopped at multipipe.pl line 102. C:\premchai21\perl>` So, from the error message, it seems as if I have a bad file descriptor. I can't imagine why, though. I think... I think I will add some debugging statements and try again.	[reply] [d/l]