Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hi, I am trying to run a set of commands 8 times- Ex: counts.sh 8 times stats.pl 8times How do i fireup the scripts that will take up 16 processors instead of waiting one to finish and then next to go on. Both system and exe functions dont seem to do that. Please help me!

Replies are listed 'Best First'.
Re: loop over to utilize all cpus
by BrowserUk (Patriarch) on Sep 17, 2010 at 15:41 UTC

    use threads; my @counters = map async( sub{ system 'counts.sh'; } ), 1 .. 8; my @staters = map async( sub{ system 'stats.sh'; } ), 1 .. 8; $_->join for @counters, @staters;

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
Re: loop over to utilize all cpus
by salva (Canon) on Sep 17, 2010 at 15:58 UTC
Re: loop over to utilize all cpus
by dasgar (Priest) on Sep 17, 2010 at 14:56 UTC

    If you're running through a loop using system and exe, you're single threaded. That means you'll only use one processor.

    If you want to use more processor cores, you'll want to use fork or threads. Of course, these will be randomly assigned to a processor core by the OS. It looks like you might be on a *nix platform. If so, you can look up the man page on numactl to see about binding a process to specific core(s).

      Here's something I wrote recently to fork 6 processes at a time (on more than 6 jobs):
      #!/usr/bin/perl use strict; use warnings; use Parallel::ForkManager; my $pm = Parallel::ForkManager->new( 6 ); my %kids; local $SIG{ INT } = sub { print "\nCaught interrupt, shutting down..."; kill 9, keys %kids; $pm->wait_all_children; print "done.\n"; exit; }; # Remove child pid from running table $pm->run_on_finish( sub { my $pid = shift; delete $kids{ $pid }; } ); for my $job ( @jobs ) { # Whatever if ( my $pid = $pm->start ) { $kids{ $pid }++; print "Started $job...\n"; } else { local $SIG{ INT } = sub { $pm->finish }; system( $job ); $pm->finish; } } $pm->wait_all_children;

      (edit) looks like I didn't reply to the parent, whoops.