Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

1. Declarations:

According to threads::shared on CPAN and perlthrtut, I should be able to declare a shared var using one of the following syntaxes:

use strict; use threads; use threads::shared; # pkg cfg { package cfg; %cfg::thash : shared = (); }

or this:

use strict; use threads; use threads::shared; # pkg cfg { package cfg; %cfg::thash = (); share(%cfg::thash); }

but the 1st gives a compile err and the 2nd gives a runtime err.
By sheer luck/desperation, I tried:

use strict; use threads; use threads::shared; # pkg cfg { package cfg; %cfg::thash = (); threads::shared::share(%cfg::thash); }
which at least doesn't cause Perl to complain, but looks like a kludge to me... Is there a less wordy/neater way?

2. Assignment

Assuming a valid declaration above, I need to assign values to the shared var, which happens to be a record structure ie I need to do this:

lock(%cfg::thash); $v1="A"; $v2="1"; $cfg::thash{$v1}{$v2} ="b";
which gives the error: 'Invalid value for shared scalar '. I haven't been able to figure out a way around this yet. Really need an answer for this, as it's for a production system I'm writing.
Any good solutions gratefully received.
Chris

Replies are listed 'Best First'.
Re: Declaring/Assigning thread shared vars
by BrowserUk (Patriarch) on Nov 18, 2004 at 00:56 UTC
    gives the error: 'Invalid value for shared scalar '

    lock(%cfg::thash); $v1="A"; $v2="1"; $cfg::thash{$v1}{$v2} ="b";

    The problem is that this line of code:

    $cfg::thash{$v1}{$v2} ="b";

    is equivalent to

    ## Make the value of key 'A' an anonymous hash. $cfg::thash{A} = {}; ## add a key '1' to that anonymous hash with the value "b"; $cfg::thash{A}{1} = "b";

    The thing with shared hashes (and shared arrays), is that everything you put into them also has to be shared.

    What's more, you cannot share an array or hash that already has data in it. Or rather you can, but it will silently empty it in the process. This is (apparently) a deliberate design decision---and a bad one in my opinion as it makes using shared hashes and arrays almost impossible.

    The basis of the decision (as it was explained to me) is that if adding elements to a shared hash automatically shared the things you put in it, then it would have to be a recursive process scanning through anything in the things you added sharing them. Ie. If you add a reference to an existing deeply nested datastructure (eg. a HoAoHoA...), then adding that reference to a shared hash, would require the entire nested structure to be traversed and all it's sub elements shared.

    This was deemed likely to take too long. It was easier to issue an uninformative error message and force the user to do the traverse. Except of course, this kind of complex processing is difficult to get right, almost impossible in user space, and exactly the kind of thing that should be done by a piece of write-it-once, get-it-right-once, tight (XS) library code.

    This is why I tend to avoid sharing compex datastructures in my threaded code. It renders autovivication impossible to use, which make using them messy and awkward.

    Anyway, you can do what you are trying to do by sharing the intermediate elements individually and building the stricture up. However, due to the use (and limitations of) perl prototypes on the threads::shared::share() function, you cannot use it on anonymous hashes or arrays directly. Instead, you have to assign a ref to them to a temporary scalar and then apply share() to that.

    This will do what you asked about:

    use strict; use threads; use threads::shared; my %thash : shared; ## Share the base hash my $temp = {}; ## Get a reference to an anon. hash in a temp var share( $temp ); ## share() the temp var $thash{A} = $temp; ## assign it into the structure $thash{A}{1} = 'b'; ## At this point you have what you asked for.

    Examine what is said, not who speaks.
    "Efficiency is intelligent laziness." -David Dunham
    "Think for yourself!" - Abigail        "Time is a poor substitute for thought"--theorbtwo
    "Memory, processor, disk in that order on the hardware side. Algorithm, algorithm, algorithm on the code side." - tachyon
      If the limitation is on shared()'s prototype, you can get perl to ignore it by calling it as &shared( ... )

        It's a good point, and a good thought, but it doesn't help (much) unfortunately.

        Whilst disabling the prototype allows me to pass an anonymous hash directly to share(), it still causes any contents in that hash to be discarded.

        It does mean that you can share and assign an empty anonymous hash directly into the parent shared hash, which avoids the temp var at the slight inconvenience of obscurity, but it still doesn't make up for the lack autovivification :(


        Examine what is said, not who speaks.
        "Efficiency is intelligent laziness." -David Dunham
        "Think for yourself!" - Abigail        "Time is a poor substitute for thought"--theorbtwo
        "Memory, processor, disk in that order on the hardware side. Algorithm, algorithm, algorithm on the code side." - tachyon
Re: Declaring/Assigning thread shared vars
by ikegami (Patriarch) on Nov 18, 2004 at 00:08 UTC

    share was imported into the current package (main?), but it was never imported into the cfg package, so shared is not available there unless you give the fully qualified name.

    Solution 1

    use strict; use threads; use threads::shared; { package cfg; use threads; use threads::shared; %cfg::thash : shared = (); }

    Solution 2

    use strict; use threads; use threads::shared; # Don't need no "package cfg", since we're # specifying cfg:: in the variable name. %cfg::thash : shared = ();

    I don't know about your other problem. I'm not familiar with multithreading in Perl.

      Actually, (unless I'm missing something,) when I tried both of those, they both gave me a syntax error at the line containing the attempt to share the hash...
Re: Declaring/Assigning thread shared vars
by Anonymous Monk on Nov 18, 2004 at 01:47 UTC
    Guys, thanks very much for the answers.
    As far as Qn 1 goes, i know I've solved it before, but don't have access to the code (it's in another ctry+company).
    As for Qn 2, very informative answer. I kind of imagine these record structs as a tree with the top level hash at the top, so if I accessed anything lower, given you have to traverse the 'tree' from the top down to find it, I assumed it would be shared anyway if you share the root of the 'tree'.
    I agree the implementation of shared could (should) be better, thus avoiding the trick you showed me. It's not very elegant ;-) , but that's not your fault of course.
    Thx again.
    Chris
Re: Declaring/Assigning thread shared vars
by Anonymous Monk on Nov 18, 2004 at 06:53 UTC
    More Help Reqd:
    Thx to BrowserUK, managed to progress, but stuck on this bit:

    $socket_errmsg = ""; ($socket_errmsg, $send_socket) = get_sending_socket( $isp, $ra +nk, $cfg::rad_srvrs{$isp}{$rank}{server}, $cfg::rad_srvrs{$isp}{$rank}{acctport} + ); + # Success ... if( !$socket_errmsg ) { # Connect succeeded, so set isp/server values $cfg::rad_srvrs{$isp}{$rank}{conn_status} = 'U'; $cfg::rad_srvrs{$isp}{$rank}{sckt} = $send_socket; # 'In +valid value for shared scalar ' . . . sub get_sending_socket { #blah,blah # Create socket $socket_errmsg = ""; $send_socket = ""; $send_socket = IO::Socket::INET->new(PeerAddr => $server, PeerPort => $port, Proto => "udp", Type => SOCK_DGRAM) or $socket_errmsg = "Couldn't create socket to $server port $port: + $@\n"; return($socket_errmsg, $send_socket); }

    The 'Invalid value for shared scalar ' error only occurs for the line marked. Thought I understood the prev explanation eg managed to extend shared hash down another level, but I'm foxed with this, especially as the preceding line has no problem. Something to do with socket's rtn value?
    NB: had the non-threaded version working ok, so I know the socket call is fine...

      The line in question is trying to assign $send_socket to a shared structure. $send_socket is a IO::Socket::INET object. Internally, that is a hash, probably containing other nested data structures.

      You would need to share that hash, and everything in it to be able to assign it to a shared hash. You cannot.

      There are two reasons for this.

      1. You cannot share a bless reference (object).

        This is because blessing, involves associating (through internal linking (termed "magic")), the data pointed at by that reference with code (the object methods). But Perl cannot run code across threads.

      2. Because if you could share that reference, it would be emptied of it's contents.

        An object without it's instance data is not going to work.

      I have seen a reference that says that you can share Perl's object's across threads. The trick (apparently) is to re-bless the (copy of the) object handle available in the (other, non creating) threads where you want to use it.

      This is a fine idea, but for it to work, you would first have to deep copy all of the contents of the blessed reference into a previously shared hash. You also have to copy any and all nested data structures referenced from it into new, shared equivalents.

      I have tried to do this for some fairly simple objects, but it is very, very, very hard to do at the Perl level. Discovering the structures, and their classes for something as complex as an IO::Handle::INET object would be fraught with problems.

      Basically what I am saying is that I cannot do what you are trying to do, and I know of noone who has succeeded in doing it.

      If you decide to go ahead and try it anyway, I think that you are very much on your own:

      • With how to even ebgin to tackle the problem.
      • With trying to get assistance if things don't work.

      Very few people will even consider trying to render assistance if you tell them that you are trying to share objects between threads. Even fewer will be responsive to bug reports against modules if you tell them you are encountering problems with their modules, when trying to share objects across threads.

      I wish it wasn't so, but it is.

      Good luck.


      Examine what is said, not who speaks.
      "Efficiency is intelligent laziness." -David Dunham
      "Think for yourself!" - Abigail        "Time is a poor substitute for thought"--theorbtwo
      "Memory, processor, disk in that order on the hardware side. Algorithm, algorithm, algorithm on the code side." - tachyon
        Thx mate. You should write the Advanced version of perlthrtut http://www.perldoc.com/perl5.8.0/pod/perlthrtut.html Actually, I've managed to re-design that bit, so I'll be creating sockets within threads, instead of shared. Cheers Chris