I'm on Linux and there is a cron process that writes at regular times the time window of validity of some forecast data for a number of geographical tiles.
Each tile has a corresponding JSON written by the cron process where, for example, its time window is written.
The data generated by the process are 4D arrays saved to ASCII files with regular structure, so that value(i,j,k,t) data are queried with direct access with seek(), upon calculation of byte start of value(i,j,k,t), through a function that has (i,j,k,t) as input.
At the moment, the process consists in looking for the tile where a given point falls and then read the JSON of such tile when making the query.
I wonder if there is a way to preload all the JSON files into a hash, and then update them when they change upon cron process execution.
Here below are parts of code, so that maybe it is possible to understand the situation. In practice, I'd like to cache the sub _get_tile_info()


get_data();

#-------------
sub get_data {

    my %args = @_;
    my @coords = @{$args{Coords} || []};

    my @tiles_and_ids = _get_tile_and_ids(%args);

    foreach my $point (@tiles_and_ids) {
        my $data = _extract_ts(Point=>$point,WS2D=>1,WD2D=>1,TEMP2D=>1);
    }

}

#----------------------
sub _get_tile_and_ids {

    my %args = @_;

    my @coords = @{$args{Coords} || []};

    foreach my $pair (@coords_lonlat)

        my ($status,$tile,$ii,$jj) = _find_tile_and_ids(X=>$xx,Y=>$yy);
        push @results,$tile,$ii,$jj;

    }

    return \@results;

}

#-----------------------
sub _find_tile_and_ids {

    my %args = @_;

    my $x = $args{X};
    my $x = $args{Y};

    # Find tile

    ....

    my ($icell,$jcell) = _find_cell(X=>$x,Y=>$y,Xmin=>$xll_tile,Ymin=>$yll_tile,Dxy=>$info{dxy});

    return('',$tile,$icell,$jcell);

}

#----------------------
sub _find_cell

    my %args = @_;

    my $x = $args{X};
    my $y = $args{Y};
    my $xmin = $args{Xmin};
    my $ymin = $args{Ymin};
    my $dxy = $args{Dxy};

    my $ii = floor(($x - $xmin) / $dxy) + 1;
    my $jj = floor(($y - $ymin) / $dxy) + 1;

    return ($ii,$jj);

}

#----------------
sub _extract_ts {

    my %args = @_;
    my $point = $args{Point} || die;

    my ($tile,$ii,$jj) = ($point->1,$point->2,$point->3)
    my %file = (
        WS2D => "$tile/ws3d.dat",
        TEMP2D => "$tile/temp3d.dat",
    );

    my $tile_info = _get_tile_info(Tile=>$tile);

    ....

}

#-------------------
sub _get_tile_info {

    my %args = @_;
    my $tile = $args{Tile};

    my $json_file = "$tile/info.json";
    my $tile_info = read_file($json_file);

    return $tile_info;

In reply to Re^2: Caching files by Anonymous Monk
in thread Caching files by Anonymous Monk

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.