I recently wrote some simple multi-process code at Re^7: Multiprocess - child process cannot be finished successfully. Yes, the "fake" Windows PID from a Perl fork is negative and my wait statement accounts for that. Code shown does run on Windows and should run also on Unix. This demo code worked better than I thought it would - meaning that sleep in each sub process worked and did not interfere with each other. I am not sure how sleep() is implemented on Windows, but it worked better than expected!

However, it sounds like just using threads is the best way for your compute bound project. The less data you share between threads (ideally nothing that is r/w), the better. Threads can get complicated if there is a lot of sharing going on.

This is a code fragment from some code years ago... The program has about 70,000 input strings. For each input string, it is desired to know which of the other input strings are "close enough" according to some complex rules. For each input, a regex is generated that is run against all other inputs. This is a NxN algorithm. For 70K inputs it took ~1.5 hours. I have a 4 core machine. Running 4 threads, execution time was something like 3.8x (can't get to exactly 4.0, but that is a very good result). So anyway execution time went to ~20 minutes and that was "good enough" and I stopping improving things.

Anyway see below for an example of parallelizing a number cruncher job.

### This is a non-runnable code fragment
### only to show a general idea of threads pulling work
### from a common input queue and putting results on a
### common output queue.

use Threads::Queue;

my @all_inputs;  global data for all threads as read only

### Worker threads #####

my $thread_limit = 4;

my @threads;
push @threads, threads->create(sub{DoWork($workQueue, $doneQueue)}) fo
+r 1 .. $thread_limit;


foreach my $input ( @all_inputs )  #init the work queue with all input
+s
{
    $workQueue->enqueue($input);
}

$workQueue->enqueue(undef) for 1 .. $thread_limit;  #"work finished" m
+arkers
                                                    #each thread will 
+give up
                                                    #when it sees an u
+ndef

$workQueue->end();

$_->join() for @threads;  #waits for all threads to finish!

print "END of threading...\n";

####  Get results off of  Queue

my @results;
while ($doneQueue->pending() && ($_ = $doneQueue->dequeue()) )
{
   push (@results, $_); #results are pointers to array (AoA)
}

sub DoWork
{
   my ($workQueue, $doneQueue) = @_;
   
   while (my $input = $workQueue->dequeue()) 
   {   
      return unless defined ($input); #this ends this thread's job!!
      
      if ($input =~ m|/|)
      {
         $doneQueue -> enqueue([$input,'SKIPPING THIS ENTRY!']);
         next;
      }
      
      my $regex = get_regex_patterns ($input); 
      $regex =~ s/\(|\)//g;  #captured values not needed, only yes/no
        
      my @matches = grep{ m/$regex/ and $_ ne $input}@all_inputs;
 
      push (@matches,'') if @matches==0;
      
      $doneQueue -> enqueue([$call,@matches]);
   }
}
[download]