stevehicks has asked for the wisdom of the Perl Monks concerning the following question:

Hi

I am writing a small app that listens to UDP datagrams sent to it from different sources and matches them up. It has a couple of threads between which it needs to share hash elements, however, I can't seem to update the hashes properly.

- the main thread listens to the messages from 3 different sources. For each mesaage it creates an entry in a dedicated hash for the given source.

- the second thread wakes up and is supposed to look at the three dedicated hashes and match up messages from each.

I am only dealing with the messages from one of these sources for now, until I can get it working.

In "main" I declare the hash for the "orc" messages and assign a "dummy" element to it that consists of a key, and a value that is another hash :

my %orcPriceHsh:shared=1; my %signatureHsh:shared = ("DE00084695946" => "0000000000.000000"); $orcPriceHsh{"0000000000"} = \%signatureHsh;

I then call the sub that listens to the messages across the network :

handleMessages($configHsh{General}{ListenPort},%orcPriceHsh);

...and spawn a thread that is supposed to look at the nested hash :

$transitCalcThread=threads->new(\&calcTransitTimes, $XmlRpcClient,$itr +sQueue,%orcPriceHsh); $transitCalcThread->detach;

in handleMessages() I assign the passed nested hash to a var "%oph" :

sub handleMessages { my $port=shift(); my %oph=@_;

...and whenever I "hear" an "orc" message I create a shared hash and add it to the top-level hash : (signature, is for example "DE0008469594621896219.525, and timeStmap is, for example "1282132286.326053", nixSeconds is, for example "1282132286,")

my %sHsh:shared = ($signature => $timeStamp); $oph{$nixSeconds} = \%sHsh;

I can dump the nested hash inside this function ok :

msg("in handleMessages() oph :"); dump(%oph);

gives :

12:51:32 : dump of oph in handleMessages() : ( 1282132292, { # tied threads::shared::tie "DE0008469594621896219.526" => "1282132292.211252", }, 1282132287, { # tied threads::shared::tie "DE00084695946218362192" => "1282132287.43417", }, 1282132286, { # tied threads::shared::tie "DE00084695946218662192" => "1282132286.326053", }, 1, undef, 0, { # tied threads::shared::tie DE00084695946 => "0000000000.000000", }, 1282132290, { # tied threads::shared::tie "DE00084695946218362192" => "1282132290.969117", }, )

but even tho the thread running calcTransitTimes() can see the initial "dummy" element it cannot see the others. This code in said function :

sub calcTransitTimes { my $XmlRpcClient=shift(); my $itrsQueue=shift(); #my %orcPriceHsh=shift(); my %args=@_; ... msg("in calcTransitTimes() args :"); dump(%args);

gives me :

12:51:32 : in calcTransitTimes() args : ( 1, undef, 0, { # tied threads::shared::tie DE00084695946 => "0000000000.000000", }, )

I know there is probably a very simple answer but I have burned a load of time on this already and I have a rapidly approaching deadline to meet. Any help would be greatly appreciated : * }

Replies are listed 'Best First'.
Re: Help !! problems sharing a nested hash across threads
by zentara (Cardinal) on Aug 18, 2010 at 12:58 UTC
      Hi I'll give those threads a read and try to work thru it, I would like to pursue shared nested hashes across threads for this from a performance perspective Many Thanks Steve
Re: Help !! problems sharing a nested hash across threads
by roboticus (Chancellor) on Aug 18, 2010 at 12:57 UTC

    stevehicks:

    I had this same problem a little while ago. The problem is that each of the hash elements is a variable, so each element in the hash must be shared--you can't simply share the top level of the hash to share all the items inside it. I can't tell you a simple way around this, though. I switched strategies for my problem and simplified the data structure so I didn't have to try to manage the housekeeping of sharing all the hash elements.

    ...roboticus

      Hi I originally used a message queues to get the messages into the other thread however I then had to vacuum them up off the queue and stick them in the hashes prior to matching. It was an easy way to do it but didn't work too well when message rates reached 150 a second. There are no doubt other methods but using shared hashes seems like a good choice thanks : ) steve
Re: Help !! problems sharing a nested hash across threads
by locked_user sundialsvc4 (Abbot) on Aug 18, 2010 at 15:16 UTC

    I would seriously re-evaluate the strategy of using multiple threads in this way.   If both of them have to update the same shared hash, then it is fairly certain that the two threads will, in fact, be running lock-step most of the time.   (Even in the presence of multiple CPUs, the two threads are in a mutual-exclusion state with respect to one another.   Therefore, why have multiple threads?)

    It would probably be much more efficient to simply have one thread doing this, even if it has nothing else to do.   A queue can be used to give the thread requests to do whatever it does.

    I often see a multi-threaded scheme used when a program has to accept inputs from multiple sources.   A new thread is created to service each source.   (In extremely bad examples, a new thread is sometimes created to service each request, in what I call “the flaming arrow approach.”)   In practice, it is probably better to have one thread that does nothing but to listen for inputs from all of these sources (at once), and which then queues the requests for processing by (one or more) service threads.   The completed outputs are then placed onto another queue, for delivery by (one...) output thread.

Re: Help !! problems sharing a nested hash across threads
by BrowserUk (Patriarch) on Aug 18, 2010 at 20:02 UTC