in reply to Re^4: Win32::Process output
in thread Win32::Process output

Let me see if I understand you correctly.

A few questions:


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
RIP an inspiration; A true Folk's Guy

Replies are listed 'Best First'.
Re^6: Win32::Process output
by Anonymous Monk on Nov 04, 2010 at 01:01 UTC

    Pretty much and much more succinct, except by 'already using threads' i meant that the solution seems to be using threads (as per your demonstration). Originally I was just going to run each executable one at a time and I'm pretty sure I could do that in the way you've explained, but I think it would be possible to speed up the process with threads. 20 was just a guesstimate, and really what I need is 'run the number of simultaneous threads that won't make my little laptop choke and die'

    I have anywhere from 1000 to 5000 (ballpark) commands to run. Each is the same command just with a different workstation name passed as a command line option each time. The application itself doesn't provide any sort of facility that i know of for passing multiple hostnames at once and the timeout option it provides doesn't actually work.

    The executable is marimba's runchannel.exe and the result is one of: 1) timeout (check with Net::Ping prior to trying and I'm assured ICMP doesn't bounce anywhere) 2) Success 3) Successful Connection, Operation Unnecessary 4) Unable to connect 5) Completely hang and never respond. I need to be able to report the first 4 results along with the workstation name, and the small number that fall into #5 I can safely ignore for the time being, but those are the ones that are causing me difficulties to script because they never return (and I've run this from commandline manually and using another tool and the same thing happens). I was able to run this once using Win32::Process->Wait() but all of the output went to stdout and I couldn't find a way to log it with the workstation name (i.e. if i change the process to cmd /c foo.exe all it outputs to file is a list without the workstation names)

      Try this:

      #! perl -slw use strict; use threads; use threads::shared; use Thread::Queue; my $semSTDOUT :shared; sub tprint{ lock $semSTDOUT; my $tid = threads->tid; print "[$tid] ", @_; } sub worker { my( $Qin, $Qout, $timeout ) = @_; ## Read a work item while( my $wkstn = $Qin->dequeue ) { my $timeout = $timeout; ## local copy my $pid :shared; ## Start the command and read the output my $thr = async { ## Replace the following with your command $pid = open my $pipe, '-|', "perl.exe runchannel.pl -w $wk +stn" or warn and next; my @output = <$pipe>; close $pipe; return join '', @output; }; sleep 1 until $pid; ## Wait until it starts ## Wait until it stops or times out sleep 1 while kill 0, $pid and --$timeout; unless( $timeout ) { kill 3, $pid; $Qout->enqueue( "$wkstn: timed out" ); $thr->join; next; } ## Que the output $Qout->enqueue( "$wkstn: " . $thr->join ); } ## ensure the main thread terminates $Qout->enqueue( undef ); threads->detach; } our $T //= 20; our $TIME //= 10; our $FILE //= 'wkstns.txt'; my( $Qwork, $Qresults ) = map Thread::Queue->new, 1 .. 2; ## Start the workers async( \&worker, $Qwork, $Qresults, $TIME ) for 1.. $T; ## Feed the queue async { open LIST, '<', $FILE or die $!; while( <LIST> ) { chomp; $Qwork->enqueue( $_ ); ## Ensure the queue doesn't runaway sleep 1 while $Qwork->pending > $T; } close LIST; ## ensure the workers terminate $Qwork->enqueue( (undef) x $T ); }->detach; ## Read & display the results for( 1 .. $T ) { chomp, tprint "R:$_" while defined( $_ = $Qresults->dequeue ); } __END__ c:\test>869283 -T=10 -TIME=5 -FILE=wkstns.txt [0] R:95.9.151.223: ok [0] R:215.80.171.135: ok [0] R:2.100.176.147: timed out [0] R:83.113.70.64: timed out [0] R:40.136.118.150: timed out [0] R:244.225.154.198: timed out [0] R:119.132.101.39: timed out [0] R:116.135.68.101: ok [0] R:219.42.173.83: timed out [0] R:178.81.107.42: timed out [0] R:7.208.47.21: ok [0] R:177.75.46.81: ok [0] R:39.193.89.80: ok [0] R:102.138.106.76: ok [0] R:42.245.57.254: ok [0] R:128.190.112.96: ok [0] R:176.108.201.231: ok [0] R:183.96.201.13: timed out [0] R:32.39.179.220: ok [0] R:53.177.236.84: ok [0] R:16.63.152.211: timed out [0] R:125.104.95.167: timed out [0] R:119.55.196.199: ok [0] R:131.213.66.174: timed out [0] R:126.249.64.145: ok [0] R:242.29.38.82: timed out [0] R:75.99.62.141: ok [0] R:170.154.206.74: ok [0] R:107.242.145.15: timed out [0] R:65.229.51.140: ok [0] R:181.239.230.146: timed out [0] R:185.234.52.92: ok [0] R:8.203.202.26: timed out [0] R:17.204.97.169: timed out [0] R:75.230.6.187: timed out [0] R:215.239.135.110: ok [0] R:180.173.240.130: timed out [0] R:63.246.36.69: timed out [0] R:216.192.44.175: timed out

      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.

        Thanks, this seems to be a good shell for what I need to do (though I'll probably wear out the Perl book figuring it out). A few questions:

        1. What is the purpose of $semSTDOUT ? It looked like the idea was to create a shared handle for file output but does this do that? Is $semSTDOUT a reserved variable for a shared handle (semaphore?) to STDOUT?

        2. What does threads->tid do? It seems to just print a '0' for all of the calls

        3. If I understand, the "async( \&worker, $Qwork, $Qresults, $TIME ) for 1.. $T; " part sets up 20 separate worker threads as part of a queue, and each worker thread reads names from a queue that is fed names from a 'master thread' that reads the file? Does the queue itself maintain the integrity (i.e. 3 worker threads ask for the next name from the queue, does the queue get locked for each call to prevent duplication?)

        Is the outer 'master thread' that feeds the names from the file into the queue strictly necessary?