Given your estimated data requirements, I would suggest not trying to optimize this. The attempt to optimize is going to be a needless complexity. The complexity of managing several processes is going to require maintainance work, if performance is not critical I would avoid that work.

But suppose that performance matters. Well then my solution would be to write a module which takes requests for a data element and returns just that data. Inside that module if it is not initialized, or if it was initialized from an older file than you see out there, then do the do. Otherwise return the data from the memoized file.

And now use mod_perl. Now if the page is heavily hit, on most requests that value will be memoized already and so it merely has to check that the file hasn't been updated and then return the current value. Sure, the data isn't shared between processes, but you get most of the way to ideal performance very easily. And if the memory requirements do get out of hand, then you can revisit the design later quite easily since the necessary code is divided between a configuration file and a simple module.

In reply to Re (tilly) 1: CGI and static data... by tilly
in thread CGI and static data... by dragonchild

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.