Dear Monks

The attached program shows a comparison of 3 ways I could use to replicate a multi-dimensional hash between child servers. I currently use the 2nd method now. The master hash is in the parent global memory, and the children manage their sub-hash by copying between global and local memory. The times are very fast, but because the global memory must be share locked on copy and exclusive locked on update, the best time possible is desired.

What's surprising to me is that to copy the entire hash is much faster than copying the sub-hash. We don't want to do that because in production the master hash can get quite large and the child should not touch anything but their copy of the sub-hash. For example, the last line of the output, the global hash is 16,000K bytes and the sub-hash is 65K bytes.

While the program is bulky, the 3 different techniques are between the "Lock/Unlock" comments. I ran this program with several different versions of perl, and on several different machines and architectures. The sample output is proportional to all, except when running the perl5.8.8 version the "Replicate sub-Hash" was about 1/2 the time of the "Loop Copy", but still more than "Replicate Entire".

So the question is, am I using the best method, or are there other better ways to replicate/copy a sub-hash?

#!/usr/local/bin/perl5.12.1 

use strict;
use Time::HiRes qw( gettimeofday );

print "\t    Replicate Entire\tLoop Copy\tReplicate sub-Hash\n","-" x 
+70,"\n";
my $Lcnt = 1_000;
for (my $i=30; $i<=256; $i+=32 )
{    ## Populate the Hash
    my %Account = (); keys( %{ $Account{$i} } ) = $i;
    for my $cnt ( 0 .. $i )
    {    for my $id ( 0 .. $i )
          {    $Account{"$cnt"}{"$id"}="x" x (int(rand($i))+6); }
     }
##    Lock the global hash %Account and Replicate the entire hash
    my $tm = gettimeofday;
    for ( 0 .. $Lcnt )        ## Just to get a big enough number to co
+mpare
            {  my %TAccount = %Account; }
    $tm = sprintf("%.7f",(gettimeofday-$tm)/$Lcnt);
##    Unlock the global hash %Account

##    Lock the global hash %Account and loop copy the local hash from 
+the global hash of hashes
    my $tm1 = gettimeofday;
    for ( 0 .. $Lcnt )        ## Just to get a big enough number to co
+mpare
    {    my %TAccount = (); keys( %TAccount ) = $i;
        foreach my $key ( keys %{ $Account{$i} } )
         {  $TAccount{$key} = $Account{$i}{$key}; }
           }
    $tm1 = sprintf("%.7f",(gettimeofday-$tm1)/$Lcnt);
##    Unlock the global hash %Account

##    Lock the global hash %Account and replicate the local hash from 
+the global hash of hashes
    my $tm2 = gettimeofday;
    for ( 0 .. $Lcnt )        ## Just to get a big enough number to co
+mpare
            {  my %TAccount = %{ $Account{"$i"} }; }
    $tm2 = sprintf("%.7f",(gettimeofday-$tm2)/$Lcnt);
##    Unlock the global hash %Account

    print $i, ":", scalar %Account, "\t$tm\t$tm1\t$tm2\n";
 }

1;

[download]
            Replicate Entire    Loop Copy    Replicate sub-Hash
--------------------------------------------------------------------
30:23/32        0.0000673       0.0001909       0.0001010
62:44/64        0.0001366       0.0003928       0.0002065
94:70/128       0.0002118       0.0005497       0.0003087
126:86/128      0.0002938       0.0008210       0.0004337
158:119/256     0.0004036       0.0010650       0.0005504
190:134/256     0.0005288       0.0011924       0.0006958
222:148/256     0.0005881       0.0014451       0.0007889
254:163/256     0.0006681       0.0017491       0.0009370

Thank you

"Well done is better than well said." - Benjamin Franklin

In reply to How best to replicate/copy a hash of hashes by flexvault

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.