CKCJim has asked for the wisdom of the Perl Monks concerning the following question:

I'm hoping I have missed a simple fix to this problem using Win32::Process, but I've done a lot of research and haven't found what I need.

I'm working on a "report engine". The main program queries the DB to retrieve a list of reports to be executed. With that information it builds the needed command line strings, which are then executed as threads using Win32::Process. The problem I'm having is not being able to capture the return from the command line for each thread and redirect it to a logfile. We really need to be able to capture any error messages to help us resolve issues faster and easier.

Here's a simple program file named ThreadLogTest.pl illustrating what I'm trying to accomplish with the redirection.

#!/usr/bin/perl
use strict;
use Win32;
use Win32::Process;

my $ProcessObj;
my $perl_command = '';

print "Starting thread logging test\n";

$perl_command = 'C:\AS_v5.8.6_Perl\bin\perl.exe D:\FinancialReports_Pe
+rlSource\PDS\output_test.pl >MyLog.txt 2>&1';

print "\n$perl_command\n";

Win32::Process::Create($ProcessObj,"C:\\AS_v5.8.6_Perl\\bin\\perl.exe"
+,$perl_command,1,CREATE_NO_WINDOW,".");

print "Thread logging test completed\n";

exit;

[download]

The output_test.pl code called by ThreadLogTest.pl is

#!/usr/bin/perl
print "\n\nThis is a test!\n\n";
exit;

[download]

The batch file that starts the whole process is

::Test thread logging

@echo STARTED %time%

Perl D:\FinancialReports_PerlSource\PDS\PDS_SURE\ThreadLogTest.pl >Thr
+eadLogTest.txt 2>&1 

@echo COMPLETED %time%

@Pause

[download]

What I need is to have the output of output_test.pl written to the MyLog.txt file. That doesn't happen. I do get the following output in ThreadLogTest.txt

Starting thread logging test

C:\AS_v5.8.6_Perl\bin\perl.exe D:\FinancialReports_PerlSource\PDS\outp
+ut_test.pl >MyLog.txt 2>&1
Thread logging test completed

[download]

Nothing is written to MyLog.txt. If I copy the command line from ThreadLogTest.txt and execute it as a command then "This is a test" does get written to MyLog.txt, so I know the command line is properly formatted to redirect the output.

I really hope someone has used Win32::Process in a similar fashion, as it seems to be the perfect solution to what we want to accomplish if we can overcome this logging issue with the threads.

Thanks to everyone for your time!

Replies are listed 'Best First'.
Re: Redirecting Output From Command Line in Threads
by ikegami (Patriarch) on Jan 28, 2011 at 21:05 UTC

    CreateProcess expects a program and its arguments — not some command for some shell — so ">MyLog.txt" and "2>&1" are being passed to Perl which passes them to your script where you completely ignore them.

    What you want is for a cmd shell to execute the string as shell command. That's not gonna happen if you don't run a cmd shell. Executing the following should do the trick: 

    cmd /c "...the shell command..."

    [download]

    You could also do the redirection yourself, thus avoiding having to load a shell. 

    use IPC::Open3 qw( open3 );

open(local *TO_CHLD,     '<', 'nul')       or die;
open(local *FR_CHLD,     '>', 'MyLog.txt') or die;
open(local *FR_CHLD_ERR, '>', 'nul')       or die;

my $pid = open3('<TO_CHLD', '>FR_CHLD', '>FR_CHLD_ERR',
   'C:\AS_v5.8.6_Perl\bin\perl.exe',
   'D:\FinancialReports_PerlSource\PDS\output_test.pl',
);

waitpid($pid, 0);  # To wait for child to finish.

    [download]

    Update: Added alternative.

[reply]
[d/l]
[select]
Re: Redirecting Output From Command Line in Threads
by cdarke (Prior) on Jan 29, 2011 at 16:31 UTC
    Win32::Process::Create creates processes, not threads. You might think that this is semantics, but the distinction is important - you will just confuse everyone if you talk about threads when you mean processes. If you are not sure of the difference, threads are several concurrent paths of execution within the same process, and require specialised techniques (not to mention the odd pagan incantation).

    Meanwhile, back at your code, ikegami suggested some solutions, although you have a couple of problems with your code.

    Your $perl_command has single back-slashes in it, which 'escapes' the character which follows (gives it a special meaning, or removes a special meaning if it had one). You did it right in the second parameter to Win32::Process::Create by using \\.

    You should always test your return value when you can - if you get the chance to trap an error then take it!

    Finally, you might find it easier to use the comspec environment variable to give you the path to the shell.

    Here are my suggestions with modifications adapted to run on my system (you will have to alter the paths, watch those \):
    #!/usr/bin/perl
use strict;
use Win32;
use Win32::Process;

my $ProcessObj;
my $perl_command = '';

print "Starting thread logging test\n";

$perl_command = "$ENV{comspec} /c C:\\perl\\bin\\perl.exe C:\\output_t
+est.pl >MyLog.txt 2>&1";

print "\n$perl_command\n";

my $result = Win32::Process::Create($ProcessObj,$ENV{comspec},$perl_co
+mmand,1,CREATE_NO_WINDOW,".");

if ($result == 0) {
    die "Error: ",Win32::FormatMessage( Win32::GetLastError() ),"\n";
}

print "Thread logging test completed\n";

    [download]
    By the way, you shouldn't really need a shell to do this redirection for you. The C/C++ Win32 API CreateProcess has a struct within it which includes handles for stdin, stdout and stderr of the child process. It is a great shame that the implementor of Win32::Process::Create chose to omit these.
[reply]
[d/l]
[select]

      Thanks for the valuable assistance. I'm not a Win32 guy by trade, so I'm still trying to absorb much of this info. Someone else I work with reminded me of the issue of threads vs. processes, and I am aware of the difference. As you say, it's semantics, but very important to avoid confusion, so please accept my apologies.

      I have applied your suggestions, and now my code is generating a log file for each "child" process. That addresses our primary need. I'm still looking at other options, as a separate log file for each process isn't where we really want to be, but at least we can move forward from here. In the long term I hope to come up with a solution that allows us to launch a large number of essentially simultaneous processes and have the command line output all logged to the same file, but if that isn't doable we will have to live with what we have right now.

      Again, thanks for your help.

[reply]

Back to Seekers of Perl Wisdom