Catching STDIN and STDERR on the fly

szabgab has asked for the wisdom of the Perl Monks concerning the following question:

I would like to write a script that can run an external application and handle its STDERR and STDOUT on the fly while supplying STDIN. My script and the external application should not block each other. I also need to make sure the two output channels are captured separately.

I looked around on CPAN and found Capture::Tiny to be one of the best tools for capturing the STDOUT and STDERR and it also list a lot of other modules that do similar tasks. I just could not figure it out yet how can I do what I wanted. Anyone here with an example for such task?

Update

Oh yes, and it needs to work on Windows, Linux and Mac OSX.

The end user of this project will be Padre - and this seems to be the major blocking before we can add an interactive debugger.

Comment on Catching STDIN and STDERR on the fly

Replies are listed 'Best First'.
Re: Catching STDIN and STDERR on the fly by jettero (Monsignor) on Dec 02, 2009 at 20:19 UTC
See IPC::Open3. Should solve the problem nicely — although it may require some thought. There are also really nice POE solutions for this, particularly if you want to do something event based instead of polling the inputs/outputs. -Paul	[reply]
Re: Catching STDIN and STDERR on the fly by ikegami (Patriarch) on Dec 02, 2009 at 22:40 UTC
It's hard. You need to avoid deadlocks. For example, you could be blocked reading from the child's STDOUT (pipe empty) when the child is blocked sending data to STDERR (pipe full). You could use some help from the OS. Unix uses a file-handle based system: IO::Select. Windows uses a more general and flexible event based system: `WaitForMultipleEvents`. `CreateFile` can be told to signal an event when a handle needs servicing. You could use threads if Perl is built with thread support. You could make the handles non-blocking and poll them. You could still block writing to the child's STDIN, though. You could use IPC::Run. IIRC, it has some support for this kind of I/O, but it's inefficient in Windows (since it attempts to model the unix system). It's interface is very simple if you know in advance what you want to send to the child (non-interactive), and IPC::Run3's is even simpler (but that's all it can do).	[reply] [d/l] [select]
Re: Catching STDIN and STDERR on the fly by Khen1950fx (Canon) on Dec 02, 2009 at 20:42 UTC
IO::NestedCapture looks flexible enough for your needs.	[reply]
Re: Catching STDIN and STDERR on the fly by repellent (Priest) on Jan 12, 2010 at 05:58 UTC
I'll submit IPC::Exe for shameless consideration :) It's lightweight in that it only has core dependencies, but is relatively new in the IPC arena.	[reply]
Re^2: Catching STDIN and STDERR on the fly by Anonymous Monk on Jan 12, 2010 at 07:25 UTC
How does it compare to IPC::System::Simple?	[reply]
Re^3: Catching STDIN and STDERR on the fly by repellent (Priest) on Jan 13, 2010 at 05:15 UTC
IPC::System::Simple is certainly easier to learn and use for simple use cases. It works great for capturing output of a known outcome (e.g. you know the output length or can expect what it looks like). But when you're not sure what the output could be, or what the size of the output is, it's generally not a good idea to capture everything into a variable for fear that it may consume too much memory. And if you need to be able to fine tune how processes spawn and fork from each other (e.g. fork a process already in the "background" so one can monitor the other), or to pipe the output of one process to another just like in a Unix shell (e.g. bash/csh \| pipe) while still being interactive like IPC::Open3, or to specify some Perl code that will only run in the child process prior to exec-ing without worrying about too much boilerplate code like `if (fork) { .. } else { .. }` ... for more flexibility, there's IPC::Exe. The perldoc provides some complex examples of what would traditionally require more than a few external scripts to achieve.	[reply] [d/l]