in reply to thread save calling an external command

need to process the standard output and standard error.

Do you need them as separate streams? How are you processing them? Could you show the code where you are processing the output?

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
In the absence of evidence, opinion is indistinguishable from prejudice. Not understood.
  • Comment on Re: thread save calling an external command

Replies are listed 'Best First'.
Re^2: thread save calling an external command
by Paul.Unix (Novice) on Jun 24, 2016 at 08:57 UTC
    The idea is to have parallel processes running to avoid long running processes block processing other items. I am mainly interested in the error output of the external commands. The exit codes of the external command are not suitable to tell what went wrong.
[reply]

      If you want both stdout and stderr (and don't mind them mixed together:

      #! perl -slw
use strict;
use threads;

sub thread {
    my $output = '';
    open CMD, '-|', q[ perl -E"say( qq[$$:saying $_] ), warn( qq[$$:wa
+rning $_\n] ), sleep(1) for 1 .. 4" 2>&1 ] or die $!;
    $output .= $_ while <CMD>;
    close CMD;
    return $output;
}

printf "Got\n'%s'\n", $_->join for map threads->create( \&thread ), 1 
+.. 4;

__END__
C:\test>t-junk
Got
'33168:warning 1
33168:warning 2
33168:warning 3
33168:warning 4
33168:saying 1
33168:saying 2
33168:saying 3
33168:saying 4
'
Got
'30208:warning 1
30208:warning 2
30208:warning 3
30208:warning 4
30208:saying 1
30208:saying 2
30208:saying 3
30208:saying 4
'
Got
'15844:warning 1
15844:warning 2
15844:warning 3
15844:warning 4
15844:saying 1
15844:saying 2
15844:saying 3
15844:saying 4
'
Got
'15032:warning 1
15032:warning 2
15032:warning 3
15032:warning 4
15032:saying 1
15032:saying 2
15032:saying 3
15032:saying 4
'

      [download]

      If you only want stderr:

      #! perl -slw
use strict;
use threads;

sub thread {
    my $output = '';
    open CMD, '-|', q[ perl -E"say( qq[$$:saying $_] ), warn( qq[$$:wa
+rning $_\n] ), sleep(1) for 1 .. 4" 2>&1 >nul ] or die $!;
    $output .= $_ while <CMD>;
    close CMD;
    return $output;
}

printf "Got\n'%s'\n", $_->join for map threads->create( \&thread ), 1 
+.. 4;

__END__
C:\test>t-junk
Got
'34456:warning 1
34456:warning 2
34456:warning 3
34456:warning 4
'
Got
'32700:warning 1
32700:warning 2
32700:warning 3
32700:warning 4
'
Got
'19172:warning 1
19172:warning 2
19172:warning 3
19172:warning 4
'
Got
'22176:warning 1
22176:warning 2
22176:warning 3
22176:warning 4
'

      [download]
      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
      In the absence of evidence, opinion is indistinguishable from prejudice. Not understood.
[reply]
[d/l]
[select]
        Thanks for this example. It got me going. I have chosen to use backticks while the thread has to wait on the completion of the external command. My conclusion, do not use IPC::Run3 or Capture::Tiny in multi threaded scripts.
[reply]

In Section Seekers of Perl Wisdom