Re: Do I need threads?

delay.pl:

use strict;
use warnings;

die "Can't open $ARGV[1] for write.\n"
    if !open(FH, ">>$ARGV[1]");

print FH "$$) Doing something with $ARGV[0]\n";
sleep rand 10;
print FH "$$) Finished running.\n";
[download]

wrapper.pl:

use strict;
use warnings;

my $log = 'logfile.txt';

unlink $log if -e $log;
for (1..10) {
   system("perl delay.pl $_ $log &");
}
[download]

logfile.txt (after random run):

395) Doing something with 1
395) Finished running.
401) Doing something with 4
401) Finished running.
403) Doing something with 5
403) Finished running.
413) Doing something with 10
413) Finished running.
399) Doing something with 3
399) Finished running.
407) Doing something with 7
407) Finished running.
409) Doing something with 8
409) Finished running.
397) Doing something with 2
397) Finished running.
405) Doing something with 6
405) Finished running.
411) Doing something with 9
411) Finished running.
[download]

Comment on Re: Do I need threads? Select or Download Code

Replies are listed 'Best First'.
Re^2: Do I need threads? by BrowserUk (Patriarch) on Dec 21, 2011 at 06:07 UTC
How does this answer the OPs requirement to obtain the output from concurrent runs of a preexisting executable? With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. The start of some sanity?	[reply]
Re^3: Do I need threads? by locked_user sundialsvc4 (Abbot) on Dec 21, 2011 at 15:13 UTC
An interesting question. My interpretation of the requirement was that “at the end of the day, we want 10 servers to have updated themselves,” and so, my thought is to: first, immediately fire a command to every one of them at once, asking every one of them at once to `cvs update` themselves and then to send back a notification, of success or of failure, by some appropriate means. The script that issued all of the simultaneous commands, without having waited for any of them to complete, then simply waits for 10 final-status messages to arrive. You’re just waiting for your proverbial mailbox to fill up, and heck, maybe you literally use e-mail to do it. Obviously, this approach would create quite a load (so to speak...) on the internal network as every one of the servers attempted to do a checkout at precisely the same time. But that “quite a load” might actually be quite reasonable. Now, having said that, yes this is clearly also a task that could be handled by forking a bunch of shell scripts ... you could even literally do the job in the shell using facilities like `'&'` on the command-line. Because the various forked processes are just issuing a command and then loafing off, sipping mint juleps while waiting for the remote machine to do its work. There is, as they say, TMTOWTDI™ in this case, all of them rather uncomplicated. No matter how you decide to tackle it, if the approach you are considering feels complicated (not just “unfamiliar”), then there is probably an easier way to do it... that ought to be the litmus-test.
Re^4: Do I need threads? by BrowserUk (Patriarch) on Dec 21, 2011 at 16:00 UTC
my thought is to: first, immediately fire a command to every one of them at once, asking every one of them at once to cvs update themselves and then to send back a notification, of success or of failure, by some appropriate means.... heck, maybe you literally use e-mail to do it. But now you're inventing a new process. He already has a process. One that works. Whatever mechanisms he uses -- he mentions a "shell script" which generally means invoking one or more executables -- it currently does one at a time and waits for the process to finish logging the progress as it goes and the checking the log locally for errors. Whilst kicking the jobs off in parallel using '>logN &' is a very simple extension of what he has now, it has the caveat of now needing to institute some mechanism by which it can wait for all the jobs to finish so the logs can be checked; or add the checking into each of the background processes plus some mechanism for notification of failures; etc. etc. Alternatively, it is easy to convert most most existing shell scripts to Perl scripts. That's the basic raison d'etre for Perl's existence. It is easy to envisage his current shell script looking something like (ignore the syntax, csh isn't my thing): foreach $webserver ( `cat servers` ) { $result = `ssh server cvs update ... `; if( echo $result \| grep Error ) { echo 'problem with server $webserver' } } [download] and easily converting that to something like: #! perl my @webservers = <>; for my $job ( @webserver ) { my $result = `ssh $job cvs update ...`; print "Server $job had errors" if $result =~ m[Error]; } [download] And it is a trivial extension of that, to do the same thing concurrently using threads: #! perl use threads; my @webservers = <>; my @threads; for my $job ( @webserver ) { push @threads, async{ my $result = `ssh $job cvs update ...`; print "Server $job had errors" if $result =~ m[Error]; }; } $_->join for @threads. [download] It's just the path of least resistance really. With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. The start of some sanity?	[reply] [d/l] [select]
A reply falls below the community's threshold of quality. You may see it by logging in.