in reply to Re^6: Avoid Locking Entire Hashes
in thread Avoid Locking Entire Hashes

The number of elements in @order in each thread is "weird". For example, 6028, 11159, 22541 and 36060. It should be 17576, 17576, 17576 and 17576.

my @order = shuffle keys %hash;
should be
my @order; { lock %hash; @order = shuffle keys %hash; }

It's the test harness that's not thread-safe.

Replies are listed 'Best First'.
Re^8: Avoid Locking Entire Hashes
by BrowserUk (Patriarch) on Jun 15, 2011 at 04:57 UTC

    Indeed, but the final death knell is that using a mirror structure of individual sems is 3 times slower than locking the entire hash!

    Lock the hash:

    #! perl -slw use strict; use Data::Dump qw[ pp ]; use Time::HiRes qw[ time ]; use List::Util qw[ shuffle ]; use threads; use threads::shared; our $START //= 'aaa'; our $END //= 'zzz'; my %hash :shared = map{ $_ => 0 } $START .. $END; my $start = time; $_->join for map{ async{ my @order; { lock %hash; @order = shuffle keys %hash; } for ( @order ) { lock %hash; ++$hash{ $_ }; } } } 1 .. 4; printf "Lock hash took %.3f seconds\n", time() - $start; my @fails = grep $_ != 4, values %hash; warn @fails . " fails\n" if @fails; __END__ C:\test>909437-1 Lock hash took 1.093 seconds

    Locking scalars:

    #! perl -slw use strict; use Data::Dump qw[ pp ]; use Time::HiRes qw[ time ]; use List::Util qw[ shuffle ]; use threads; use threads::shared; our $START //= 'aaa'; our $END //= 'zzz'; my %sems :shared = map{ my $n :shared = 0; $_ => \$n } $START .. $END; my %hash :shared = map{ $_ => 0 } $START .. $END; my $start = time; $_->join for map{ async{ my @order; { lock %hash; @order = shuffle keys %hash; } for ( @order ) { lock ${ $sems{ $_ } }; ++$hash{ $_ }; } } } 1 .. 4; printf "Lock scalars ref took %.3f seconds\n", time() - $start; my @fails = grep $_ != 4, values %hash; warn @fails . " fails\n" if @fails; __END__ C:\test>909437-3 Lock scalars ref took 4.667 seconds

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      Indeed. I recognised that your solution is better as soon as I saw it, at laest in terms of memory usage. For short locks, it's equally slow compared to locking the whole hash, though.
      >perl ikegami.pl Lock scalars ref took 1.648 seconds >perl ikegami.pl Lock scalars ref took 1.714 seconds >perl ikegami.pl Lock scalars ref took 1.702 seconds >perl browseruk.pl Lock scalars ref took 1.731 seconds >perl browseruk.pl Lock scalars ref took 1.543 seconds >perl browseruk.pl Lock scalars ref took 1.601 seconds
        it's equally slow compared to locking the whole hash, though.

        Hm. How many cores do you have?


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.