in reply to Non-blocking I/O woes

I think $select->can_read() is blocking but not in the way you are expecting. can_read will return a handle when the end of file is reached as well. A feature I did not expect is that can_read will return an empty list, the blocking abruptly stopped, when a signal is caught by a signal handler. This means, after the SIGCHLD handler is executed the code execution returns from can_read with an empty list. This empty list means the end of your while loop and script.

Inside your spawn sub, when you are forking by using open you are not exiting the forked child process. The child will merrily return from the spawn sub and act as a proud new parent. Two forked processes reading the same STDIN is a recipe for bizarre behavior.

I think it would also be easier to use open to call the "script" program directly instead of forking and then calling the script with exec. If you use open to call the external program then you don't need to worry about exiting the forked process after exec.

Here is a version I messed with tonight. IPC is always interesting...

#!/usr/bin/env perl use warnings; use strict; use IO::Handle qw(); use IO::Select qw(); use File::Temp qw(tempfile); #--------------------------------------------------------------------- +------- # CHILDREN my $CHILD_SCRIPT_PATH = './child.pl'; my $Select_inputs = IO::Select->new(); my %Child_of_handle; sub spawn_child { my ( $script_input ) = @_; my ( $tempfh, $tempname ) = tempfile(); print $tempfh $script_input; close $tempfh; # flush to disk my $pid = open my $kidoutput, "$CHILD_SCRIPT_PATH <$tempname |"; die "fork failed: $!" unless defined $pid; # Turns off buffering as well? Without this the child's # output comes all at once when it exits. $kidoutput->blocking( 0 ); $Child_of_handle{ $kidoutput->fileno } = [ $pid, $tempname ]; $Select_inputs->add( $kidoutput ); return; } # Kill all child processes. sub kill_procs { warn "KILLING CHILD PROCESSES\n"; kill 9, $_->[0] for values %Child_of_handle; } # Cleanup data and temporary input file associated with the child's pi +pe handle. sub cleanup_child { my ( $handle ) = @_; my $child_ref = $Child_of_handle{ $handle->fileno } or die "Failed to lookup child handle $handle"; my ( $pid, $tempfile ) = @$child_ref; delete $Child_of_handle{ $handle->fileno }; $Select_inputs->remove( $handle ); unlink $tempfile or warn "Failed to delete temp file $tempfile:\n$ +!"; return; } #--------------------------------------------------------------------- +------- # COMMANDS / PROMPT my ( %Commands, $Prompt ); sub prompt_state { my ( $input ) = @_; my ( $first, @args ) = split /\s+/, $input; unless ( $first ) { warn sprintf "Please enter a command: %s\n", join q{, }, keys %Commands; return; } my $cmd = $Commands{ $first }; unless ( $cmd ) { warn "Unknown command: $first\n"; return; } # I don't use arguments but you see it is possible. $cmd->( @args ); return; } my $run_input = q{}; sub run_state { my ( $input ) = @_; unless ( $input eq q{.} ) { $run_input .= $input . qq{\n}; print q{>}; return; } $Prompt = \&prompt_state; unless ( $run_input ) { warn "IGNORING EMPTY INPUT\n"; return; } spawn_child( $run_input ); $run_input = q{}; return; } %Commands = ( 'run' => sub { $Prompt = \&run_state; print q{>}; }, 'exit' => sub { kill_procs(); exit 0; }, ); #--------------------------------------------------------------------- +------- # MAIN LOOP # STDIN fileno is 0 sub STDIN_NUM() { 0 } STDOUT->autoflush( 1 ); # for the run prompt's > to display properly.. +. $Select_inputs->add( \*STDIN ); $Prompt = \&prompt_state; $SIG{'INT' } = sub { kill_procs(); exit 1; }; $SIG{'CHLD'} = 'IGNORE'; SELECT_LOOP: while ( 1 ) { my @ready = $Select_inputs->can_read or die 'select returned an empty list'; # User input is read from the prompt... if ( grep { $_->fileno == STDIN_NUM } @ready ) { # STDIN is line-buffered my $line = <STDIN> || q{}; chomp $line; $Prompt->( $line ); next SELECT_LOOP; } # Child process input (this is output from the child's perspective +) my $next = shift @ready; if ( $next->eof ) { # If EOF is reached we are done with this child process. cleanup_child( $next ); next SELECT_LOOP; } my $pid = $Child_of_handle{ $next->fileno }[0]; my $output = do { local $/; <$next> }; # slurp! print map { "$pid:$_\n" } split /\n/, $output; }
edit: wrapped code in a readmore tag

Replies are listed 'Best First'.
Re^2: Non-blocking I/O woes
by ikegami (Patriarch) on Jan 11, 2011 at 05:58 UTC

    my $line = <STDIN> || q{};

    You introduced bugs by moving away from sysread.

    First, you've introduced buffered IO. How can the select system call know that Perl has data waiting to be read in a buffer?

    Second, you're blocking until a newline is received, and that could be an indefinitely long wait.

      The bug doesn't happen because perl has STDIN line-buffered. Select won't notice STDIN has read data until you press enter. This also means there will always be a newline to read for <STDIN>.

      sysread will give you the unbuffered activity you expect but select will not. I think this is where you are mistaken. This means select will still be blocking until you press enter. I was able to get select to trigger on a single character by using tcsetattr to disable line-buffering (aka canonical mode) for the terminal. (code at the end of the post)

      edit: Just to be clear about what this implies, this leads me to believe that line-buffering happens at a lower level than the perl interpreter. This answers your rhetorical question:

      How can the select system call know that Perl has data waiting to be read in a buffer?

      It is my hypothesis that the buffer you speak of happens at the system level and not inside the perl interpreter. Using stdio.h functions like getc and getline, you must read input through the terminal's internal buffer while in canonical mode. The read system call (via perl's sysread) does not use this higher-level terminal line buffer.

      I have no idea how this works in windows.

      #!/usr/bin/env perl use warnings; use strict; use IO::Select; use IO::Handle; use Inline 'C'; my $select = IO::Select->new( \*STDIN ); disable_canon_mode() if @ARGV; # Experiment by commenting out the above line and giving command-line # arguments. while ( 1 ) { $select->can_read; # block until new input warn "CAN_READ STDIN\n"; my $in; if ( @ARGV ) { sysread STDIN, $in, 1; print "Char read: $in\n"; } else { $in = <STDIN>; print "Line read: $in\n"; } } __END__ __C__ #include <termios.h> #define STDIN 0 void disable_canon_mode () { struct termios tio; if ( isatty( STDIN ) == 0 ) { return; } /* Turn off canonical mode, meaning read one char at a time. Set timer to 0, to wait forever. Minimum chars is 1. */ tcgetattr( STDIN, &tio ); tio.c_lflag &= ~ICANON; tio.c_cc[ VTIME ] = 0; tio.c_cc[ VMIN ] = 1; tcsetattr( STDIN, TCSANOW, &tio ); }

        The bug doesn't happen because perl has STDIN line-buffered. Select won't notice STDIN has read data until you press enter. This also means there will always be a newline to read for <STDIN>.

        Line-buffering only makes sense for output handles, and STDIN is not an output handle.

        Secondly, you don't only use readline (<>) on STDIN.

        See Re^3: Malfunctioning select() call on a FIFO for a demonstration of the problem.

        It is my hypothesis that the buffer you speak of happens at the system level and not inside the perl interpreter.

        Sometimes, yes. Not always.

        I have no idea how this works in windows.

        Not very well, since select only works on sockets in Windows.

Re^2: Non-blocking I/O woes
by ikegami (Patriarch) on Jan 11, 2011 at 05:51 UTC

    A feature I did not expect is that can_read will return an empty list, the blocking abruptly stopped, when a signal is caught by a signal handler.

    select returns on error, and that includes interruption by signal (EINTR).

Re^2: Non-blocking I/O woes
by dwalin (Monk) on Jan 12, 2011 at 16:23 UTC
    Thanks! That was it - somehow I never expected can_read() to return with an empty handle array upon receiving a signal. But it does and it really explains everything: the silent exit and the erratic behavior and all that. Now that I got this mystery solved I'm happy and productive again. :)