rastoboy has asked for the wisdom of the Perl Monks concerning the following question:

Greetings Brothers, I've been banging my head against the wall trying to nail down good IPC technique for a couple months now, and just when I think it all makes sense I can't quite test my understandings successfully. In particular, my research indicates that it's essential basically to use IO::Select to find out if read and write handles are ready for reading and writing before using them. In particular I've studied some nice code here:

http://www.perlmonks.org/?node_id=151886

But what's happening when I try my own implementation (including *writing* to the child process) is IO::Select seems to let the write file handle sneak through as ready for reading! Here is my "server" code:

#!/usr/bin/perl print "What is your name?\n"; my $name = prompt_for_input(); print "What is your mission?\n"; my $mission = prompt_for_input(); print "What is your favorite color?\n"; my $color = prompt_for_input(); print "Your name is $name, your mission is $mission, and your favorite + color is $color. Have a nice day!\n"; sub prompt_for_input { local ($|) = (1); my $resp = <STDIN>; chomp $resp; return $resp; }
Simple enough, right? Here is my "client":
#!/usr/bin/perl use strict; use warnings; use diagnostics; use IPC::Open3; use IO::Select; my @answers = qw/galahad seekthegrail bluenored!/; my( $wtr, $rdr, $err ); use Symbol 'gensym'; $err = gensym; my $pid = open3( $wtr, $rdr, $err, 'server.pl' ); print "IN: $wtr OUT: $rdr ERR: $err\n"; print "PID was $pid\n"; my $sel = new IO::Select; $sel->add( $wtr, $rdr ); my $line; foreach my $readhandle ( $sel->can_read(1) ) { print "readhandle is $readhandle \n"; next if $readhandle != $rdr; print "past next\n"; my $len = sysread $readhandle, $line, 4096; if( not defined $len ) { die "Error from child: $!\n"; } print $line; write_answers(); } waitpid( $pid, 0 ); sub write_answers { foreach my $writehandle ( $sel->can_write ) { print "write handle is $writehandle\n"; return if $writehandle != $wtr; my $answer = shift @answers; print "writing $answer\n"; print $writehandle $answer . "\n"; } }
So what happens is very strange:
rasto@frodo:~/work$ ./client.pl IN: GLOB(0x880fe78) OUT: GLOB(0x880fe88) ERR: GLOB(0x880fe28) PID was 6834 readhandle is GLOB(0x880fe78) readhandle is GLOB(0x880fe88) past next write handle is GLOB(0x880fe78) writing galahad rasto@frodo:~/work$
So there are two things happening here that I don't understand. 1. Why did IO::Select brings GLOB(0x880fe78) up as ready for reading at all, when it was opened as the write (input) filehandle? To paraphrase Basil Fawlty, "What's the point of the bl**dy thing if it doesn't work when you really need it?" lol.

And 2. I don't understand why the program terminates after writing the answer to the first question. I suspect the two things are interrelated, and I feel like I'm *so close* to really understanding how this process is supposed to work--any input on understanding this would be greatly appreciated!

Replies are listed 'Best First'.
Re: Weirdness with IO::Select and IPC::Open3
by ikegami (Patriarch) on Mar 21, 2011 at 02:00 UTC

    Why did IO::Select brings GLOB(0x880fe78) up as ready for reading at all, when it was opened as the write (input) filehandle?

    select actually signals when a file handle needs to be serviced. In addition to "ready to read" for can_read, that includes EOF and error conditions.

    In this case, the OS is ready to communicate to you that you can't read from a write-only handle.

    In general, you want two select objects, one for readers and one for writers. If you want to wait for either, you'd use select instead of can_*.

    my ($r, $w) = IO::Select::select($readers_sel, $writers_sel, undef);

    I don't understand why the program terminates after writing the answer to the first question.

    You only call can_read once. You appear to be missing a loop.

      In this case, the OS is ready to communicate to you that you can't read from a write-only handle.

      Ikegami, could you by any chance point me to documentation that explains what you're talking about, here? The more I think about it, the less sense it makes, and I'm dying for a doc that just tells me how this system works. I don't even know what "system" I'm talking about, here! :-)

      I mean, it seems to me I tell IO::Select to watch these two file handles, one of which has been specified as the STDOUT filehandle for the child process--I just can't grasp how that filehandle should ever show up in can_read().

      I know I'm wrong, but I'd like to get properly educated on the subject--I feel like I'm stabbing in the dark, still, know what I mean?

        The purpose of select is to parallelise a few things including reading from the handles identified by its first argument. As such, select should return whenever sysread would return for any of those handles.

        sysread obviously returns when provided a write-only handle.

        $ perl -E' open(my $fh, ">", "file") or die $!; my $rv = sysread($r[0], $buf, 100); if (!defined($rv)) { say "Error: $!"; } elsif (!$rv) { say "eof"; } else { say "Got $rv bytes"; } ' Error: Bad file descriptor

        As such, select should do the same.

        $ perl -MIO::Select -E' open(my $fh, ">", "file") or die $!; my @r = IO::Select->new($fh)->can_read(); my $rv = sysread($r[0], $buf, 100); if (!defined($rv)) { say "Error: $!"; } elsif (!$rv) { say "eof"; } else { say "Got $rv bytes"; } ' Error: Bad file descriptor

        You're saying select should block forever in the event of an error even though sysread would not. That makes no sense.

Re: Weirdness with IO::Select and IPC::Open3
by ikegami (Patriarch) on Mar 21, 2011 at 02:48 UTC

    I don't know why you're using select for this problem. If you were to use select, you'd have

    use strict; use warnings; use IO::Handle qw( ); # For autoflush. use IO::Select qw( ); use IPC::Open3 qw( open3 ); use Symbol qw( gensym ); use constant BLK_SIZE => 64*1024; my $pid = open3( my $stdin_fh = gensym(), my $stdout_fh = gensym(), my $stderr_fh = gensym(), 'server.pl', ); $stdin_fh->autoflush(1); my $stdin_buf = ''; my $stdout_buf = ''; my $stderr_buf = ''; my $writers = IO::Select->new(); my $readers = IO::Select->new($stdout_fh, $stderr_fh); my %answers = ( "What is your name?" => 'ikegami', "What is your mission?" => 'To become the King of Halloween' +, "What is your favorite color?" => 'orange', ); while ($readers->count()) { my ($ready_readers, $ready_writers) = IO::Select::select($readers, +$writers, undef); foreach my $fh (@$ready_readers) { if ($fh == $stdout_fh) { my $rv = sysread($fh, $stdout_buf, BLK_SIZE, length($stdout_b +uf)); if (!defined($rv)) { $readers->remove($fh); # ...[ Do something in response to error from sysread. ].. +. } elsif (!$rv) { $readers->remove($fh); # ...[ Do something in response to EOF from sysread. ]... } else { while ($stdout_buf =~ s/(.*)\n//) { my $query = $1; if ($query =~ /Have a nice day!\z/) { $readers->remove($fh); print("$query\n"); } elsif ($answers{$query}) { $stdin_buf .= "$answers{$query}\n"; $writers->add($stdin_fh); } else { # ...[ Do something with unrecognised output from ST +DIN. ]... } } } } if ($fh == $stderr_fh) { my $rv = sysread($fh, $stderr_buf, BLK_SIZE, length($stderr_b +uf)); if (!defined($rv)) { $readers->remove($fh); # ...[ Do something in response to error from sysread. ].. +. } elsif (!$rv) { $readers->remove($fh); # ...[ Do something in response to EOF from sysread. ]... } else { # ...[ Do something in response to output to STDERR. ]... } } } foreach my $fh (@$ready_writers) { if (length($stdin_buf)) { # It's only safe to send one bytes unless you # somehow query the system to find otherwise. print($stdin_fh substr($stdin_buf, 0, 1, '')); # ...[ Do something if print returned an error. ]... } if (!length($stdin_buf)) { $writers->remove($fh); } } } waitpid($pid, 0); # ...[ Do something if waitpid returned an error. ]...
      Hi ikegami, thanks for all the great responses. It's going to take me a while to study it all, but I was wondering--how would you do it without IO::Select? You seem to be suggesting it is less than ideal for the purpose, but all I know is when I google the errors I get when I don't use it, everyone says to use it :-).

      Color me confused.

        If you don't care about the child's STDERR, you can send it to the parent's STDERR (using '>STDERR' for open3's third arg). Then, all you need is

        ... while (<$stdout_fh>) { chomp; if (/Have a nice day!\z/) { print("$query\n"); } elsif ($answers{$query}) { print($stdin_fh "$answers{$query}\n"); } else { # ...[ Do something with unrecognised output from STDIN. ]... } }

        You can do this because you have a strict request-response protocol.

        If you had a more asynchronous protocol, you still have lots of simpler options than select. These include IPC::Run, Expect and threads.

        when I google the errors I get when I don't use it, everyone says to use it :-).

        What errors would that be?

Re: Weirdness with IO::Select and IPC::Open3
by zentara (Cardinal) on Mar 21, 2011 at 15:07 UTC
    Hi, all those gensyms you are using have me confused in your code, but try this example and see how it works. Your timeout of 1 in can_read also looks messy.

    Also see IPC3 buffer limit problem and ioctl for a way to detect if buffers have anything in them.

    #!/usr/bin/perl # What you want to do is create # one IO::Select object with both handles, then ask it to # wait until one or both have something ready to read. # Something like this: # It's only drawback is it only outputs 1 line of bc output # so it errs on something like 234^12345 (which outputs a big number) use warnings; use strict; use IPC::Open3; use IO::Select; #interface to "bc" calculator my $pid = open3(\*WRITE, \*READ,\*ERROR,"bc"); my $sel = new IO::Select(); $sel->add(\*READ); $sel->add(\*ERROR); my($error,$answer)=('',''); while(1){ print "Enter expression for bc, i.e. 2 + 2\n"; chomp(my $query = <STDIN>); #send query to bc print WRITE "$query\n"; foreach my $h ($sel->can_read) { my $buf = ''; if ($h eq \*ERROR) { sysread(ERROR,$buf,4096); if($buf){print "ERROR-> $buf\n"} } else { sysread(READ,$buf,4096); if($buf){print "$query = $buf\n"} } } } waitpid($pid, 1); # It is important to waitpid on your child process, # otherwise zombies could be created.

    I'm not really a human, but I play one on earth.
    Old Perl Programmer Haiku ................... flash japh