What is the correct way to finish multithreaded program?

Gangabass has asked for the wisdom of the Perl Monks concerning the following question:

Hi, Monks

I have some multithreaded code which gives me warnings at the end:

Perl exited with active threads: 1 running and unjoined 2 finished and unjoined 0 running and detached

my $q  = Thread::Queue->new();
my $pq = Thread::Queue->new();

my @threads =
  map { threads->create( \&worker, $_ ) } ( 1 .. $config->{number_of_t
+hreads} );

push @threads, threads->create( \&controller );

my $mech = WWW::Mechanize->new();
$mech->agent_alias("Windows IE 6");

$mech->get( $config->{start_url} );

my @states = find_states($mech);

foreach my $state (@states) {

    process_state($state);
}

while ( $q->pending() ) {

    sleep 1;
}

foreach ( 1 .. $config->{number_of_threads} ) {

    $q->enqueue(undef);
}

while ( $pq->pending() ) {

    sleep 1;
}

$pq->enqueue(undef);

foreach my $thr (@threads) {

    if ( $thr->is_joinable() ) {
        $thr->join();
    }
}
[download]

sub process_state() is populating $q queue. Each thread contain something like this:

    while ( my $org = $q->dequeue() ) {

        parse_org( $org, $mech );
        $pq->enqueue($org);
    }
[download]

Thanks. Roman

Comment on What is the correct way to finish multithreaded program? Select or Download Code

Replies are listed 'Best First'.
Re: What is the correct way to finish multithreaded program? by moritz (Cardinal) on May 05, 2014 at 08:46 UTC
While I'm not an expert on threads, I believe this is the problem: `foreach my $thr (@threads) { if ( $thr->is_joinable() ) { $thr->join(); } }` [download] You only join the threads that have already finished; instead you should join them all, i.e. wait for them to finish. Perl 6 - the future is here, just unevenly distributed	[reply] [d/l]
Re^2: What is the correct way to finish multithreaded program? by Gangabass (Vicar) on May 05, 2014 at 11:18 UTC
Hmm... According to docs: $thr->is_joinable() Returns true if the thread has finished running, is not detached and has not yet been joined. In other words, the thread is ready to be joined, and a call to $thr->join() will not block. Previous block of code (`while (..) {..}`) used to wait threads to finish (I think). But may be I'm wrong :-(	[reply] [d/l]
Re^3: What is the correct way to finish multithreaded program? by moritz (Cardinal) on May 05, 2014 at 11:37 UTC
Previous block of code `(while (..) {..})` used to wait threads to finish (I think). No, it waits for the queue to be empty, which isn't quite the same thing. Perl 6 - the future is here, just unevenly distributed	[reply] [d/l]
Re^4: What is the correct way to finish multithreaded program? by Gangabass (Vicar) on May 05, 2014 at 12:18 UTC
Re^5: What is the correct way to finish multithreaded program? by BrowserUk (Patriarch) on May 05, 2014 at 15:49 UTC
Re^5: What is the correct way to finish multithreaded program? by Anonymous Monk on May 05, 2014 at 13:08 UTC
Some notes below your chosen depth have not been shown here
Re^5: What is the correct way to finish multithreaded program? by Anonymous Monk on May 05, 2014 at 12:24 UTC
Re^3: What is the correct way to finish multithreaded program? by Anonymous Monk on May 05, 2014 at 11:35 UTC
Previous block of code (while (..) {..}) used to wait threads to finish (I think). But may be I'm wrong :-( Maybe, it depends on more code than shown I think :) safest way to wait for threads to finish is to actually wait for them to finish :) blockingly `$_->join for threads->list`	[reply] [d/l]
Re: What is the correct way to finish multithreaded program? by Anonymous Monk on May 05, 2014 at 08:05 UTC
Perl exited with active threads: 1 running and unjoined 2 finished and unjoined 0 running and detached Can you describe what this means? Read more... (1471 Bytes)	[reply] [d/l]


Think about Loose Coupling
	PerlMonks