Re^2: capture stdout and stderr from external command

I am running these external commands in Windows and I want to run 60 of the 5000 external commands that I have at any given point of time. When one of the external commands is done I need to run another such that I have 60 external commands executing at any given time. Thats the reason why I have been using perl's forkmanager which forks the process and limits them to the max procs (60 in my case). Now all I need is to get the stdout and stderr of these external commands which run from the forked child process.

my $max_procs = 60;
my $pm = new Parallel::ForkManager($max_procs);
foreach my $child ( 0 .. $#cmds ) {  
    my $pid = $pm->start($cmds[$child]) and next;
    # This is where I need to get the stdout and stderr.
    # cmds can be external command which can be windowsexecutable.exe 
+$args (for example perl.exe script.pl $arg1 $arg2)
    system("cmds"); 
    my $Result = $? >> 8;
    $pm->finish($Result); # pass an exit code to finish
  }

  $pm->wait_all_children;
[download]

The above sample code set the $max_procs to 60 and forks 60 child process each executing one system command. All I need is to get the stdout and stderr of the executable executed using the system command into some perl variable

Comment on Re^2: capture stdout and stderr from external command Download Code

Replies are listed 'Best First'.
Re^3: capture stdout and stderr from external command by zwon (Abbot) on Nov 12, 2011 at 03:33 UTC
Have a look onto POE::Wheel::Run, it allows you to run a lot of children and capture their outputs. You have to learn a bit about POE first though.	[reply]
Re^3: capture stdout and stderr from external command by zentara (Cardinal) on Nov 12, 2011 at 11:15 UTC
All I need is to get the stdout and stderr of the executable executed using the system command into some perl variable Your problem is that forking puts the $cmd into a different $pid, and if you put your stdout and stderr into a perl variable, it won't be seen in the parent. You will need to open some pipes from the children back to the parent, to write back your returns. Or, you could use threads or some other form of IPC, like shared memory segments. See forking with Storable and IPC::ShareLite I'm not really a human, but I play one on earth. Old Perl Programmer Haiku ................... flash japh	[reply]
Re^3: capture stdout and stderr from external command by remiah (Hermit) on Nov 12, 2011 at 02:22 UTC
Windows can also do 2>&1 like bash. So how about like this? `$pm->start($cmds[$child] . " > /tmp/$$.$child.log 2>&1")` [download] $child may be unique through commands. And collect the outputs afterwards?	[reply] [d/l]