How to split output using Expect.pm

kudra has asked for the wisdom of the Perl Monks concerning the following question:

Problem: When using Expect.pm, fatal errors generated by the program being executed do not seem distinguishable from other output by the program.

Here is a pruned example to demonstrate the problem:

my $object = Expect->spawn("program");                  # 1
my $error = ($object->expect(30, 'password:'))[1];      # 2
die "$error\n" if ($error);                             # 3
print $object "incorrectpassword\r";                    # 4
$object->soft_close();                                  # 5
[download]

An error in line 2 caused by a problem with the expect command will go to STDERR. An error generated by 'program'--such as will be generated after sending the incorrect password at line 4--will go to STDOUT, not STDERR, which is also where any data resulting from a valid run of 'program' will go.

Is there some way I can separate the output of 'program' based upon the status of 'program'? If there is any mention of such behavior in the Expect docs then it was probably in the section that was too dense for me, so please elaborate instead of pointing me to them, as I have read quite a few times.

Comment on How to split output using Expect.pm Download Code

Replies are listed 'Best First'.
(jcwren) RE: How to split output using Expect.pm by jcwren (Prior) on Aug 16, 2000 at 18:59 UTC
merlyn is too quick to jump to a conclusion here. <G> It seems that Expect combines the STDERR and STDOUT of the program together. By adding the parameter '2>errfile.tmp', kudra and I determined that the STDERR from the program will go to the 'errfile.tmp' file, and STDOUT will go where expected. The output in 'errfile.tmp' can now be processed as desired. This works quite well for her application, although it would be cooler if the output could be tied straight into a scalar, rather than having to open the file for processing. Note that 'errfile.tmp' is not an ideal name, as it's not process specific, nor garanteed. We're aware that there are some suggestions for creating unique filenames, but that wasn't the point of this node. Perl Cookbook, section 16.7, talks about redirecting I/O. Revised code: `my $object = Expect->spawn("program 2>errfile.tmp"); # 1 my $error = ($object->expect(30, 'password:'))[1]; # 2 die "$error\n" if ($error); # 3 print $object "incorrectpassword\r"; # 4 $object->soft_close(); # 5` [download] --Chris e-mail jcwren	[reply] [d/l]
RE: (jcwren) RE: How to split output using Expect.pm by tye (Sage) on Aug 17, 2000 at 08:10 UTC
That is what pipes are for! Create a pipe, prevent it from being closed on exec, get the FD of it, then use `2>&$FD` on the command line and you can read from the pipe to read the command's STDERR. Something like this; not just untested, I've never done anything like this before (: ... `{ local($^F)= 0x8000; pipe( READERR, WRITEERR ) or die "pipe: $!"; } my $fd= fileno(WRITEERR); Expect->new( "command 2>&$fd" ); close(WRITEERR);` [download] I'd probably do this using fcntl() instead of $^F so that I could have READERR closed on exec while WRITEERR stays open. To interleave the reading from the pipe with the use of Expect, it would probably be best to extend the expect() function to allow an extra file handle to be passed to the select() and return with a special code when that file handle gets some data. And if you're going to extend the expect() function, you could also extend the spawn() function so that it has an option to dup WRITEERR (instead of the new STDOUT) to STDERR. <RANT> Why the h*** does Expect do a `exec join(' ',@_)` instead of just `exec @_`? Why get passed the arguments pre-parsed if you're just going to smash them all together and force the shell to reinterpret them! If I wanted to have the shell mess up my command line, I'd pass it in as a single string, the way Larry intended!</RANT> - tye (but my friends call me "Tye")	[reply] [d/l] [select]
Re: How to split output using Expect.pm by merlyn (Sage) on Aug 16, 2000 at 15:37 UTC
I don't believe there's any way that a program that sits on a pty (like `Expect.pm`) can ever distinguish where the data is coming from. This is the blessing and curse of Unix I/O. So, no, I think you're just gonna have to consider the various things to wait for, some of which will be error-like messages coming at unexpected times. -- Randal L. Schwartz, Perl hacker	[reply]
RE: How to split output using Expect.pm by Adam (Vicar) on Aug 16, 2000 at 21:20 UTC
I would bet that you could make good use of btrott's Filehandle Filter to solve this problem in your code. You could filter STDERR do prefix all the lines with something like, "ERR: ", or better yet, do some work with caller() to get a stack trace on anything going to STDERR...	[reply]
RE (tilly) 2: How to split output using Expect.pm by tilly (Archbishop) on Aug 17, 2000 at 03:26 UTC
Unfortunately not. btrott's filter only works on output coming from your program (and even so has some limitations, as I pointed out it won't catch output from xsubs or system calls you make). kudra is interested in filtering the output of another program. However your basic idea is good, only the implementation has problems. Indeed you could pipe STDOUT and STDERR of the program through filters that would tag the output. There probably is a nice way to do it, but one silly and inefficient way is to launch a wrapper program that opens pipes to two filter programs that will tag their output, then launches your desired command using IPC::Open3, connecting it's STDIN to yours, and its STDOUT and STDERR to the two filters. The wrapper can, incidentally, use the wait call to reap the children and also get the exit code of the command you are trying to wrap.	[reply]
RE: RE (tilly) 2: How to split output using Expect.pm by Adam (Vicar) on Aug 17, 2000 at 03:35 UTC
Yeah, I realized that too... and I msged kudra, but I never quite got around to updating my post. Besides, I really like that filter and I was hoping someone would find a way to make that solution work :-)	[reply]