in reply to wait for threads to end in parallel

It seems to me that there should be a problem--thr1 could be sitting their idle while thr0->join is blocking--but I don't know how you can see that problem in the output.

If it will work for you, you don't need to create new threads. You can just use the same two threads over and over:

use strict;
use warnings;
use 5.010;

use threads;
use threads::shared;
use Thread::Queue;

my $q : shared;
$q = Thread::Queue->new;    #a Queue is thread safe, which means 
                            #threads can read from it without
                            #interfering with each other.

my @files = (
    'file1',
    'file2',
    'file3',
    'file4',
    'file5',
);


my $thread_count = 2;
my @threads;

for (1 .. $thread_count) {
    my $thread_name = "thread$_";
    push @threads, threads->create(\&do_stuff, $thread_name);
}

$q->enqueue(@files); #...and the starting gun sounds!

for (1 .. $thread_count) {
    $q->enqueue("END_OF_QUEUE");
}

for my $thr (@threads) {
    $thr->join();
}



sub do_stuff {
    my $thr_name = shift;

    while ( (my $file = $q->dequeue) ne "END_OF_QUEUE" ) {

        #$q->dequeue() blocks until there is something to retrieve
        #from the queue.

        say "$thr_name is opening file: $file";

        #doing some work:
        sleep int(rand 4);

        say "$thr_name is is done with: $file";
    }
}


--output:--
thread1 is opening file: file1
thread2 is opening file: file2
thread2 is is done with: file2
thread2 is opening file: file3
thread1 is is done with: file1
thread1 is opening file: file4
thread2 is is done with: file3
thread2 is opening file: file5
thread2 is is done with: file5
thread1 is is done with: file4

[download]

Replies are listed 'Best First'.
Re^2: wait for threads to end in parallel
by BrowserUk (Patriarch) on Mar 16, 2010 at 11:46 UTC
    It seems to me that there should be a problem--thr1 could be sitting their idle while thr0->join is blocking

    Why would thr0->join block, when he has tested thr0->is_joinable?

    $thr->is_joinable() Returns true if the thread has finished running, is not detached and has not yet been joined. In other words, the thread is ready to be joined, and a call to $thr->join() will not block.

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]
[select]

      Ah, I see. I read is_joinable() in the code, but I didn't understand its significance. So the while loop polls each thread to discover when a thread is finished processing a file.

      With some slight modifications of the op's code, the output shows that the threads do not appear to be waiting for each other (the code does get stuck in an infinite loop when the threads are done processing the files):

      use strict;
use warnings;
use 5.010;


use threads;
use threads::shared;

my @file_list = (
    'file1',
    'file2',
    'file3',
    'file4',
    'file5',
);


my $nofiles = @file_list;      #number of files 
my $currfile = 1;              #current number of file to process

my %MSG : shared;              #shared hash

my $thr0 = threads->new(\&process, shift(@file_list), 'thr0');
$currfile++;

my $thr1 = threads->new(\&process, shift(@file_list), 'thr1');
$currfile++;

while(1) {
 if ($thr0->is_joinable()) {
  $thr0->join;
        #check if there are files left to process
  if($currfile <= $nofiles){ 
   $thr0 = threads->new(\&process, shift(@file_list), 'thr0');
   $currfile++;
  }
 }

 if ($thr1->is_joinable()) {
  $thr1->join;
        #check if there are files left to process
  if($currfile <= $nofiles){
   $thr1 = threads->new(\&process, shift(@file_list), 'thr1');
   $currfile++;
  }
 }
}

sub process{
       my($file, $thr_name) = @_;

       print "$thr_name opening $currfile of $nofiles\n";

       #do some stuff
       sleep int(rand 5);

       print "$thr_name done with $currfile of $nofiles\n";
}

--output:--
thr0 opening 1 of 5
thr1 opening 2 of 5
thr1 done with 2 of 5
thr1 opening 3 of 5
thr1 done with 3 of 5
thr1 opening 4 of 5
thr0 done with 1 of 5
thr0 opening 5 of 5
thr1 done with 4 of 5
thr0 done with 5 of 5

      [download]

      Ugh. That is really icky code.

[reply]
[d/l]

In Section Seekers of Perl Wisdom