castaway has asked for the wisdom of the Perl Monks concerning the following question:

Hi peoples,

I thought my script which uses either ithreads, 5005threads or none, depending on environment, was working fine and dandy.. Until I noticed that I should have been sharing some objects (plugins) between the threads, and I wasnt, and thus the fun started.

According to the threads::shared docs:

"bless" is not supported on shared references. In the current version, "bless" will only bless the thread local reference and the blessing will not propagate to the other threads. This is expected to be implemented in a future version of Perl.
However, my tests show that this is not quite true. There are some strange things you can do, and more importantly, not do, with shared variables. It seems you can bless shared variables, for example, and the blessed object works fine in a thread.. You cant share a blessed object though, since whenever you share something, its contents get clobbered. These and other things might have been useful to put the threads::shared docs.. See test code below: 
#!/usr/bin/perl -w

use Data::Dumper;
use Storable 'freeze', 'thaw';
use threads;
use threads::shared;

use Net::Ping;

# Create hashref, set a value
my $hashref = {};
$hashref->{'mine'} = 12;

#share hashref - original value gone!
share($hashref);
print "Shared hashref: ", Dumper($hashref), "\n";

# Create object
my $ping = Net::Ping->new();
print "Object: ", Dumper($ping), "\n";

# Share object - values gone
share($ping);
print "Shared object (empty): ", Dumper($ping), "\n";

# Recreate (bless shared thing, works fine..)
$ping = Net::Ping->new();
print "Shared object (renewed): ", Dumper($ping), "\n";

# Create, share, then bless.. works..
my $newping = {};
share($newping);
$newping = Net::Ping->new();
print "Shared then bless: ", Dumper($newping), "\n";

# Use in another thread, just to check
my $thr = threads->create(\&ping_it);
#  Wait for ping to complete
$thr->join();

sub ping_it
{
    print "ping_it ", Dumper($ping);
    print "Ping localhost: ", ($ping->ping('localhost') ? "yes" : "no"
+), "\n";
    # Works!
}

# Put shared thingy in other shared thingy.. (I want(ed) a hash of obj
+ects..)

my $hashy = {};
share($hashy);

# $hashy->{'ping'} = $ping;
# print "Hash of obj: ", Dumper($hashy);
# Damn, doesnt work - Invalid value for shared scalar at ../threadtest
+.pl line 53.

my %hashy = ();
share(%hashy);

# $hashy{'ping'} = $ping;
# print "Hash of obj: ", Dumper(%hashy);
# Damn, doesnt work - Invalid value for shared scalar at ../threadtest
+.pl line 53.

# my $testping = Net::Ping->new();
# $hashy->{'ping'} = $testping;
# Can't do that either..

# $hashy->{'ping'} = freeze($ping);
# print "Hash of obj: ", Dumper(%hashy);
# Nope..

$hashy->{'ping'} = freeze(pingy->new());
print "Hash of obj: ", Dumper($hashy);
# hmm, works.. (but only if no globs or code-refs)
$thr = threads->create(\&test_me);
$thr->join();

sub test_me
{
    print Dumper(thaw($hashy->{'ping'}));
}

package pingy;
use Data::Dumper;

sub new
{
    # Create new pingy
    # Parameter: Class-Name/Reference, Properties
    my $class = shift;
    $class = ref($class) || $class;
    my $self = {'just' => 'testing'};
    bless($self, $class);

    print "Created: ", Dumper($self), "\n";
    return $self;
}

[download]
.. So, if I keep my modules free of code refs and globs, it seems I can cheat a little, use freeze/thaw, and actually share objects. Of course, no guarantee that this behaviour will work in the next version..

(All tests here done with 5.8.2, on linux)

Anyone know how, what, why and WTF?

C.

Replies are listed 'Best First'.
Re: ithreads, shared variables..
by liz (Monsignor) on Dec 24, 2003 at 09:24 UTC
    ...You cant share a blessed object though, since whenever you share something, its contents get clobbered...

    Actually, sharing a simple scalar does keep its value: 

    use threads;
use threads::shared;
my $foo = "foo\n";
share( $foo );
print $foo;
__END__
foo

    [download]

    Internally, share() is basically a tie on the reference of what is specified. This is the same approach I use with forks: oddly enough, forks doesn't have that problem, e.g. with a hash: 

    use forks;
use forks::shared;
my %foo = (bar => "foo\n");
share( %foo );
print %foo;
__END
barfoo

    [download]
    I think I'll keep this "inconsistency" in forks, but will add a little note in the pod about this. I agree this is a bug in threads::shared that should be fixed. As always, patches are welcome ;-)

    ...These and other things might have been useful to put the threads::shared docs...

    Please feel free to supply a documentation patch to p5p, or a proposal for (additional) text here (in which case I'll turn it into a documentation patch and submit it to p5p on your behalf).

    ...it seems I can cheat a little, use freeze/thaw, and actually share objects...

    FWIW, that's the approach that Thread::Queue::Any, Thread::Conveyor and Thread::Tie use under the hood (which is actually externalized into its own module: Thread::Serialize).

    ...Anyone know how, what, why and WTF?

    Look at shared.xs in your Perl distribution. I think most of the problem comes from the fact that there is internally only 1 subclass for the tieing: threads::shared::tie. Whereas in forks, I have taken the approach of creating a seperate subclass for each data type.

    Hope this helps.

    Liz

[reply]
[d/l]
[select]
      Ok.. I missed the fact that simple scalars actually keep their values when shared.. That does indeed look like a bug then, it's inconsistent.

      My suggestion for the docs would be manyfold. First, they should probably mention that shared variables are implemented using 'tie'. Second, that sharing already filled complex vars (arrays, hashes) clobbers the contents. While we're at it, the Thread::Queue documentation could mention that 'elements' in its case are just simple scalars, and not references to arrays etc. (It just mentions 'scalars' which can contain any number of things).

      So for threads::shared:
      NOTES

      sharing is implemented via tied variables, and thus all caveats applying to those should be applied here too.
      BUGS
      "bless" is not supported on shared references. In the current version, "bless" will bless the reference in each thread, but the objects in each will work independantly. This is expected to be implemented in a future version of Perl.
      sharing hashes and arrays (and refernces to such) which already contain values is not currently supported, the contents of the hash or array will be deleted. (ie what gets shared is a new, empty structure).
      structures containing code references or globs cannot be put in a shared variable.

      And for Thread::Queue:
      NOTE

      Only simple scalars can be used in Queues, complex scalars such as references and objects cannot be used.

      I've been playing around some more, and having removed code refs from my objects (and replaced with symbolic references, Muhahahaha.. ), I've gotten them shared and usable, even without using freeze/thaw. There was a bug due to a shared variable not having enough scope, which is probably also a sharing/threads bug, it was a closed variable, the functions that used it were inside the correct scope...

      On another note, I have another nit to pick with shared variables (I think you mentioned this recently somewhere..), just sharing a structure (HoH in this case) isnt enough, every single value added to it needs to be shared (if briefly) as well. This is very annoying, it would be nice to be able to specify that the thing being added is being added by value, if that makes any sense. Thus: 

      my $HoH : shared = {};

sub add_thing
{
    my $time = get_time();
    $HoH->{'one'} = &share($time);
}

      [download]
      wouldnt need the extra '&share' in the middle there.. silly really, as $time is about to go back out of scope.

      No doubt this also makes more sense if I read the source.. I'll take a look, but I'd prefer to be able to treat such things as black boxes, and have the documentation explain what they do, without taking them apart myself.

      Update: Oops, blessing propagates, but doesnt actually work.. darn it!

      C.

[reply]
[d/l]
Re: ithreads, shared variables..
by pg (Canon) on Dec 24, 2003 at 00:47 UTC
    "It seems you can bless shared variables, for example, and the blessed object works fine in a thread.. "

    But you don't call that "works", as what you want is to have it shared, but it does not give you that. The caveat stated in the document is precise, but I think it would be better to have some sample code with it, not just description in natural language.

    As you said that you tested with 5.8.2, seems that it is not fixed in 5.8.2. Plus 5.8.2 might core dump when you detach thread, I would say not to upgrade to 5.8.2, if thread is important to you.

    On the other hand, 5.8.1 thread has unreasonably large memory usage if you don't detach it, so make sure to detach your thread if you don't expect to join later.

[reply]
      Umm, either you read me wrong, or theres some misunderstanding. Objects/blessed scalars *do* work fine when shared, I can use them in more than one thread, no problem. The perldoc claims that 'bless' only blesses a thread local reference, and leaves the value in other threads, shared or not. This is plain untrue, according to my tests at least.

      Update: Oops. I didnt test enough, it seems. The bless creates an object in each thread the object is shared in, but changes to one do not reflect in the other, so I guess they're not really shared after all. </darn>

      I upgraded to 5.8.2 some time ago, while testing another thread problem, which core dumped in 5.8.1, this now works in 5.8.2. (See 5005threads -> ithreads (porting)) I hadnt used 5.8.1 much, because of the quick release of 5.8.2, and the fact that it is binary compatible with 5.8.0, unlike 5.8.1.

      C.

[reply]
Re: ithreads, shared variables..
by castaway (Parson) on Dec 26, 2003 at 20:09 UTC
    Hmm, now Im not sure if blessed variables work or not.. I got it working again, I think.. Should probably make some simple example..

    C.

[reply]

Back to Seekers of Perl Wisdom