Here is another way, using JSON to make your storage files
use strict; use warnings;
use JSON;
my $usedeltatime=0;
eval 'use time::DeltaTime;' if ($usedeltatime);
my @bigarray;
my %bighash;
doit(0,0);
doit(1,0);
doit(1,1);
sub doit {
my $usecache1=shift;
my $usecache2=shift;
deltatime('all','set') if ($usedeltatime);
if ($usecache2) {$usecache1=1;}
if ($usecache1){
open (my $cache1,'<','json1.json') or die $@;
my $save= do {local $/;<$cache1>};
close $cache1;
@bigarray= @{decode_json($save)};
deltatime('all','stage','cache1r:'.$usecache1) if ($usedeltatime)
+;
}
else {
for my $ii (1..30000) {
if (rand(100)>80) { push @bigarray,$ii; }
}
my $save = encode_json (\@bigarray);
open (my $cache1,'>','json1.json') or die $@;
print $cache1 $save;
close $cache1;
deltatime('all','stage','cache1w:'.$usecache1) if ($usedeltatime)
+;
}
if ($usecache2){
open (my $cache2,'<','json2.json') or die $@;
my $save= do {local $/;<$cache2>};
close $cache2;
%bighash= %{decode_json($save)};
deltatime('all','stage','cache2r:'.$usecache2) if ($usedeltatime)
+;
}
else {
foreach my $i (@bigarray){
foreach my $j (@bigarray) {
my $positive = $i + $j;
next if $positive > 28123;
$bighash{$positive} ++;
}
}
my $save = encode_json (\%bighash);
open (my $cache2,'>','json2.json') or die $@;
print $cache2 $save;
close $cache2;
deltatime('all','stage','cache2w:'.$usecache2) if ($usedeltatime);
+
}
print 'array:'.scalar(@bigarray)."\n";;
print 'hash :'.scalar(keys %bighash)."\n";;
deltatime('all','done','cache1:'.$usecache1.'cache2:'.$usecache2)
+if ($usedeltatime);
} # dout
I have a little routine to create delta-times between stages, you dont, so leave $usedeltatime=0; Others may show you different ways of timing, ive been using deltatime for almost 2 decades, so i used it now.
d-time:000:00:00 000:00:00->all:cache1w:0
d-time:000:00:30 000:00:30->all:cache2w:0
array:5994
hash :28079
d-time:000:00:00 000:00:30->all:cache1:0cache2:0@2017-04-09-04-33-41
d-time:000:00:00 000:00:00->all:cache1r:1
d-time:000:00:29 000:00:29->all:cache2w:0
array:5994
hash :28079
d-time:000:00:00 000:00:29->all:cache1:1cache2:0@2017-04-09-04-34-10
d-time:000:00:00 000:00:00->all:cache1r:1
d-time:000:00:01 000:00:01->all:cache2r:1
array:5994
hash :28079
d-time:000:00:00 000:00:01->all:cache1:1cache2:1@2017-04-09-04-34-11
With deltatime enabled, no caching used 29 seconds, caching just bigarray used 29 seconds, but caching both bigarray and bighash only used 1 second
YMMV
|