threads :: shared hash of hashes (array of arrays)

konstant1n has asked for the wisdom of the Perl Monks concerning the following question:

Hi Perl gurus, I've a question related to threads and threads::shared modules. Say I've N threads all working on some kind of computation. Also a thread may need to access other thread's data during the computation process. Seems that I should create a shared data structure, something like hash of arrays or hash of hashes so each thread will have its own array/hash to store results of computation and all those array/hashes will be joined into one shared structure in a form of array or hash. I wrote a small test, but I;m getting strange results... cannot figure out what I;m doing wrong on my own. /don' understand why value of step 0 is missing /

#!/usr/bin/perl
use strict;
use warnings;

use threads;
use threads::shared;
my %totals :shared;
my $stop :shared = 0;

sub thread_worker {
        my $n = shift;  # thread N
        my @args = shift;
        my %temp = ();
        print("Thread $n started: ", join(' ', @args), "\n");

        my $s=0;
        while(1) {
            my $random = rand();
            print "I'm thread $n, step $s, random num $random\n";

            # each thread has its own hash to store the results 
            # on each step of the loop we add (loop_step_number->rando
+m_value_generated) into hash
            # /me: waisting memory using temp hash, I don't know how t
+o get around syntax of share()            
            #       
            $totals{$n}->{STEPS} = $s;                  # record step
            $temp{$s}=$random;                          # add value to
+ per-thread hash 
            $totals{$n}->{HASH}= &share( \%temp );  # copy results to 
+shared structure


            $s++;
            sleep 1;
            sleep 70 if $stop;  # stopping calculation by going into l
+og sleep
        }
}


## launch threads
my @thr = ();
for(my $i = 0; $i <= 9; $i++ ) {

    $totals{$i} = &share( { STEP=>0, HASH=>0} );   # create a datastru
+cture for each thread
    $thr[$i] = threads->create('thread_worker', $i, 'ar gy ment');
    $thr[$i]->detach();
}

sleep 2;  # let's give threads some time to work
$stop =1; # stop all worker threads

# now check results of the workers
while ( my ($thread_num, $results) = each %totals ) {
     print "thread $thread_num, completed " . ($results->{STEPS} +1) .
+ " steps\n";
     for (my $st=0; $st <= $results->{STEPS}; $st++) {
        print "\t step: $st  random value: $results->{HASH}->{$st}\n";
     }
}
[download]

gives the following output

Thread 0 started: ar gy ment
I'm thread 0, step 0, random num 0.430995690710009
Thread 1 started: ar gy ment
I'm thread 1, step 0, random num 0.342638185088621
Thread 2 started: ar gy ment
I'm thread 2, step 0, random num 0.442201701532216
Thread 3 started: ar gy ment
I'm thread 3, step 0, random num 0.447242501265467
Thread 4 started: ar gy ment
I'm thread 4, step 0, random num 0.100245214606733
Thread 5 started: ar gy ment
I'm thread 5, step 0, random num 0.498271692152233
Thread 6 started: ar gy ment
I'm thread 6, step 0, random num 0.478716604019613
Thread 7 started: ar gy ment
I'm thread 7, step 0, random num 0.51481957973829
Thread 8 started: ar gy ment
I'm thread 8, step 0, random num 0.577251307739974
Thread 9 started: ar gy ment
I'm thread 9, step 0, random num 0.202572321127267
I'm thread 0, step 1, random num 0.893937211807014
I'm thread 1, step 1, random num 0.304837428123818
I'm thread 2, step 1, random num 0.0687422201476409
I'm thread 3, step 1, random num 0.00751381772303716
I'm thread 4, step 1, random num 0.447827941595705
I'm thread 5, step 1, random num 0.179563435583503
I'm thread 6, step 1, random num 0.827086767970712
I'm thread 7, step 1, random num 0.691312452635856
I'm thread 8, step 1, random num 0.626051676782907
I'm thread 9, step 1, random num 0.467705887599976
I'm thread 0, step 2, random num 0.544733674536598
I'm thread 1, step 2, random num 0.4474694747861
I'm thread 2, step 2, random num 0.0640259532175484
I'm thread 3, step 2, random num 0.936233804832352
I'm thread 4, step 2, random num 0.68358204680602
I'm thread 5, step 2, random num 0.244457205562991
I'm thread 6, step 2, random num 0.40358483213107
I'm thread 7, step 2, random num 0.838718247405392
thread 6, completed 3 steps
Use of uninitialized value in concatenation (.) or string at ./t.pl li
+ne 53.
     step: 0  random value: 
     step: 1  random value: 0.827086767970712
     step: 2  random value: 0.40358483213107
thread 3, completed 3 steps
Use of uninitialized value in concatenation (.) or string at ./t.pl li
+ne 53.
     step: 0  random value: 
     step: 1  random value: 0.00751381772303716
     step: 2  random value: 0.936233804832352
thread 7, completed 3 steps
Use of uninitialized value in concatenation (.) or string at ./t.pl li
+ne 53.
     step: 0  random value: 
     step: 1  random value: 0.691312452635856
     step: 2  random value: 0.838718247405392
thread 9, completed 2 steps
Use of uninitialized value in concatenation (.) or string at ./t.pl li
+ne 53.
     step: 0  random value: 
     step: 1  random value: 0.467705887599976
thread 2, completed 3 steps
Use of uninitialized value in concatenation (.) or string at ./t.pl li
+ne 53.
     step: 0  random value: 
     step: 1  random value: 0.0687422201476409
     step: 2  random value: 0.0640259532175484
thread 8, completed 2 steps
Use of uninitialized value in concatenation (.) or string at ./t.pl li
+ne 53.
     step: 0  random value: 
     step: 1  random value: 0.626051676782907
thread 1, completed 3 steps
Use of uninitialized value in concatenation (.) or string at ./t.pl li
+ne 53.
     step: 0  random value: 
     step: 1  random value: 0.304837428123818
     step: 2  random value: 0.4474694747861
thread 4, completed 3 steps
Use of uninitialized value in concatenation (.) or string at ./t.pl li
+ne 53.
     step: 0  random value: 
     step: 1  random value: 0.447827941595705
     step: 2  random value: 0.68358204680602
thread 0, completed 3 steps
Use of uninitialized value in concatenation (.) or string at ./t.pl li
+ne 53.
     step: 0  random value: 
     step: 1  random value: 0.893937211807014
     step: 2  random value: 0.544733674536598
thread 5, completed 3 steps
Use of uninitialized value in concatenation (.) or string at ./t.pl li
+ne 53.
     step: 0  random value: 
     step: 1  random value: 0.179563435583503
     step: 2  random value: 0.244457205562991
A thread exited while 11 threads were running.
[download]

PS: default perl 5.8.8 package on centos 4.x x86_64

Comment on threads :: shared hash of hashes (array of arrays) Select or Download Code

Replies are listed 'Best First'.
Re: threads :: shared hash of hashes (array of arrays) by gone2015 (Deacon) on Oct 18, 2008 at 00:26 UTC
When I tried some threaded code I found that the `&share()` function was next to useless for hashes and arrays. The documentation (threads::shared) appears to pretty much agree: BUGS AND LIMITATIONS When share is used on arrays, hashes, array refs or hash refs, any data they contain will be lost. I found that the trick was to declare shared arrays/hashes. Then you can put references to those shared items into other shared data structures. So, where you have: `sub thread_worker { my $n = shift; # thread N my @args = shift; my %temp = (); .... while(1) { .... $totals{$n}->{STEPS} = $s; # record step $temp{$s}=$random; # add value to per +-thread hash $totals{$n}->{HASH}= &share( \%temp ); # copy results to +shared structure .... } }` [download] I would try: `sub thread_worker { my $n = shift; # thread N my @args = shift; my %temp : shared = (); # Make shared $totals{$n}->{STEPS} = 0 ; # Nothing yet $totals{$n}->{HASH} = \%temp ; # Set ref to shared result +s .... while(1) { .... $totals{$n}->{STEPS} = $s; # record step $temp{$s}=$random; # add value to per-thread +hash # REMOVED: $totals{$n}->{HASH}= &share( \%temp ); .... } }` [download] and see if it works any better. (It did for me.)	[reply] [d/l] [select]
Re^2: threads :: shared hash of hashes (array of arrays) by Anonymous Monk on Oct 18, 2008 at 01:45 UTC
wow, what was pretty clever! thank you very much. It works as expected now. -konstantin	[reply]
Re: threads :: shared hash of hashes (array of arrays) by zentara (Cardinal) on Oct 18, 2008 at 12:34 UTC
One rule I follow is that shared hashes will only share their first level key by default. So you need to explicitly share, then define, all deeper keys manually. I usually do something like this, to give each thread it's own shared hash. Of course, you can get more clever coding, but this makes it clear. my %shash; #share(%shash); #will work only for first level keys my $numworkers = 3; foreach my $dthread(1..$numworkers){ share ($shash{$dthread}{'go'}); share ($shash{$dthread}{'progress'}); share ($shash{$dthread}{'timekey'}); #actual instance of the thread share ($shash{$dthread}{'frame_open'}); #open or close the frame share ($shash{$dthread}{'handle'}); share ($shash{$dthread}{'data'}); share ($shash{$dthread}{'pid'}); share ($shash{$dthread}{'die'}); $shash{$dthread}{'go'} = 0; $shash{$dthread}{'progress'} = 0; $shash{$dthread}{'timekey'} = 0; $shash{$dthread}{'frame_open'} = 0; $shash{$dthread}{'handle'} = 0; $shash{$dthread}{'data'} = $data; $shash{$dthread}{'pid'} = -1; $shash{$dthread}{'die'} = 0; $hash{$dthread}{'thread'} = threads->new(\&work,$dthread); } [download] I'm not really a human, but I play one on earth Remember How Lucky You Are	[reply] [d/l]
Re^2: threads :: shared hash of hashes (array of arrays) by BrowserUk (Patriarch) on Oct 18, 2008 at 13:02 UTC
See also the `share_clone()` function exported by the latest versions of threads::shared. It is essentially what `share()` should have been from the get go. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply] [d/l] [select]
Re^3: threads :: shared hash of hashes (array of arrays) by konstant1n (Novice) on Oct 22, 2008 at 22:33 UTC
Thanks. I'll keep the note.	[reply]