flexvault has asked for the wisdom of the Perl Monks concerning the following question:
Dear Monks
The attached program shows a comparison of 3 ways I could use to replicate a multi-dimensional hash between child servers. I currently use the 2nd method now. The master hash is in the parent global memory, and the children manage their sub-hash by copying between global and local memory. The times are very fast, but because the global memory must be share locked on copy and exclusive locked on update, the best time possible is desired.
What's surprising to me is that to copy the entire hash is much faster than copying the sub-hash. We don't want to do that because in production the master hash can get quite large and the child should not touch anything but their copy of the sub-hash. For example, the last line of the output, the global hash is 16,000K bytes and the sub-hash is 65K bytes.
While the program is bulky, the 3 different techniques are between the "Lock/Unlock" comments. I ran this program with several different versions of perl, and on several different machines and architectures. The sample output is proportional to all, except when running the perl5.8.8 version the "Replicate sub-Hash" was about 1/2 the time of the "Loop Copy", but still more than "Replicate Entire".
So the question is, am I using the best method, or are there other better ways to replicate/copy a sub-hash?
#!/usr/local/bin/perl5.12.1 use strict; use Time::HiRes qw( gettimeofday ); print "\t Replicate Entire\tLoop Copy\tReplicate sub-Hash\n","-" x +70,"\n"; my $Lcnt = 1_000; for (my $i=30; $i<=256; $i+=32 ) { ## Populate the Hash my %Account = (); keys( %{ $Account{$i} } ) = $i; for my $cnt ( 0 .. $i ) { for my $id ( 0 .. $i ) { $Account{"$cnt"}{"$id"}="x" x (int(rand($i))+6); } } ## Lock the global hash %Account and Replicate the entire hash my $tm = gettimeofday; for ( 0 .. $Lcnt ) ## Just to get a big enough number to co +mpare { my %TAccount = %Account; } $tm = sprintf("%.7f",(gettimeofday-$tm)/$Lcnt); ## Unlock the global hash %Account ## Lock the global hash %Account and loop copy the local hash from +the global hash of hashes my $tm1 = gettimeofday; for ( 0 .. $Lcnt ) ## Just to get a big enough number to co +mpare { my %TAccount = (); keys( %TAccount ) = $i; foreach my $key ( keys %{ $Account{$i} } ) { $TAccount{$key} = $Account{$i}{$key}; } } $tm1 = sprintf("%.7f",(gettimeofday-$tm1)/$Lcnt); ## Unlock the global hash %Account ## Lock the global hash %Account and replicate the local hash from +the global hash of hashes my $tm2 = gettimeofday; for ( 0 .. $Lcnt ) ## Just to get a big enough number to co +mpare { my %TAccount = %{ $Account{"$i"} }; } $tm2 = sprintf("%.7f",(gettimeofday-$tm2)/$Lcnt); ## Unlock the global hash %Account print $i, ":", scalar %Account, "\t$tm\t$tm1\t$tm2\n"; } 1;
Replicate Entire Loop Copy Replicate sub-Hash
--------------------------------------------------------------------
30:23/32 0.0000673 0.0001909 0.0001010
62:44/64 0.0001366 0.0003928 0.0002065
94:70/128 0.0002118 0.0005497 0.0003087
126:86/128 0.0002938 0.0008210 0.0004337
158:119/256 0.0004036 0.0010650 0.0005504
190:134/256 0.0005288 0.0011924 0.0006958
222:148/256 0.0005881 0.0014451 0.0007889
254:163/256 0.0006681 0.0017491 0.0009370
Thank you
"Well done is better than well said." - Benjamin Franklin
|
|---|