in reply to Re^2: Caching files
in thread Caching files
I created a Makefile like this:
SHELL = /bin/bash .PHONY: data data: for i in {1..10} ; do for j in {1..10} ; do echo "[$$i,$$j,$$RANDO +M]" > $$i-$$j.json ; done ; done .PHONY: simulate_cron simulate_cron: data while : ; do \ i=$$((1 + RANDOM % 10)) ; j=$$((1 + RANDOM % 10)) ; echo "[$$i +,$$j,$$RANDOM]" > tmp ; mv tmp $$i-$$j.json ;\ done .PHONY: query query: time perl query.pl .PHONY: clean clean: rm -f *.json
Now, you can run
to generate the input data and start modifying them randomly.make simulate_cron
Then, run
in a different terminal. The Perl program is the following:make query
#!/usr/bin/perl use warnings; use strict; use feature qw{ say }; use Cpanel::JSON::XS qw{ decode_json }; my %cache; for (1 .. 1000) { my @queries = map [ map int 1 + rand 10, 1, 2 ], 1 .. 50; for my $query (@queries) { my ($x, $y) = @$query; # delete $cache{$x}{$y}; # <- Uncomment to simulate no cache. my $value; if (exists $cache{$x}{$y} && (stat "$x-$y.json")[9] == $cache{$x}{$y}{last} ) { $value = $cache{$x}{$y}{value}; } else { open my $in, '<', "$x-$y.json" or die $!; $cache{$x}{$y}{last} = (stat $in)[9]; $value = $cache{$x}{$y}{value} = decode_json(do { local $/ +; <$in> })->[2]; } say "$x, $y: $value"; } }
With the delete line uncommented, it takes about 0.400s to terminate. With the line commented, it runs under 0.100s, i.e. slightly more than 4 times faster.
Notes:
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re^4: Caching files
by Anonymous Monk on Jan 24, 2020 at 17:21 UTC |