in reply to Re^16: does threads (still) memleak?
in thread does threads (still) memleak?

I just tried using the versions you mentioned, and I still see the problem arising.

That is not good news. Please raise a perlbug against thread::shared immediately.

I'm starting to think it's Dumper's fault.

That's a distinct possibility, though the maintainers might argue that they should not be able to be the cause of such a problem.

In the meantime, try using Data::Dump and see how you fare? (But please raise the perlbug report anyway, even if it alleviates your immediate problem!).


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

Replies are listed 'Best First'.
Re^18: does threads (still) memleak?
by faxm0dem (Novice) on Nov 28, 2008 at 15:30 UTC
    I think I ruled out Dumper and STDERR cache:
    #!/usr/bin/perl use warnings; use strict; use threads; use threads::shared; my %hash :shared; sub dumper { my $h = shift; my $ret = "{\n"; while (my ($k,$v) = each %$h) { $ret.= "\t$k => $v\n"; } $ret.= "}\n"; return $ret; } sub sub1 { my $str = '[s'.threads->tid()."]\n"; $str.= dumper \%hash; $str.= '[e'.threads->tid()."]\n"; return $str; } $hash{a} = 1; my $th1 = threads->new('sub1'); $hash{b} = 2; my $th2 = threads->new('sub1'); my $a = $th1->join(); my $b = $th2->join(); print STDERR $a; print STDERR $b;
    Which sometimes yields:
    [s1] { a => 1 b => 2 a => 1 b => 2 } [e1] [s2] { a => 1 } [e2]
    And I just reproduced it on perl 5.8.5 32 and 64bit linux. Sometimes I also get:
    [s1] { a => 1 b => 2 b => 2 } [e1] [s2] { a => 1 a => 1 b => 2 } [e2]
    Then again, even though we see duplicate keys, their value is correct... Maybe the problem arises when trying to cycle through all keys of the hash at the exact same time its contents are being updated: If I add a select undef,undef,undef,0.01; between the two thread spawns, I never see duplicate keys. At the worst I see normal differences between $a and $b.
      Maybe the problem arises when trying to cycle through all keys of the hash at the exact same time its contents are being updated:

      That makes a certain amount of sense in the version above with the sleeps removed. It's quite easy to see how (without locking) that the hash iterator could be corrupted. But that does not explain the earlier versions where you have sleeps that effectively orchestrate serialisation of the writing and reading of the hash.

      Yes, you should be using locking. But for locking to be the fix for the original (with sleeps) version of the code, it would mean that iterating a 2 key hash would have to take longer than not just one timeslice (in order for the other thread to have a need to lock), but longer than 1 second. And that's just silly.

      A quick test shows that Perl can iterate a two-key shared hash 10 million times in just over 4 seconds:

      use threads; use threads::shared; use Time::HiRes qw[ time ]; my $h:shared = ( 1..4 ); print time; for ( 1..1e7 ) { 1 while each %h; } print time; 1227886927.80338 1227886932.10025

      Even on a multicore box and with Data::Dumper in the mix, there's no way that the sleeping version of the code you posted could be concurrently modifying and iterating that hash.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
        Are you saying 726584 suggests a bug but 726616 doesn't? I mean in the first example the sleep is inside the sub, so executed almost at the same time for both threads. I don't see the difference.