If your data needs to be constructed fresh for every request, it doesn't make sense to store it in a file and then read it back - in fact, it would only make the whole system slower. If OTOH it's only created every 5 minutes or so, I would suggest running a script via cron to create the data.

Depending on the type of data in your hash, and the size of it, you might be better off using a DBM module or a relational database for faster lookups (data::dumper requires you to read the entire structure in memory before you can use it or filter out the results you want to see)

By the way, a "couple of thousand" hash keys might not be a problem at all - depending on how fast you can generate your data and how many requests you get, the amount of memory in the server and CPU speed. If you have some time to experiment, try the "dumb but easy" approach first (i.e. just generate the hash and filter out the results, no files or databases etc), and see if it works before you start throwing all kinds of optimization techniques at it.

You might also be able to just generate the requested data, instead of all of it - but that really depends on the algorihm used to generate it.

"What should it profit a man, if he should win a flame war, yet lose his cool?"

In reply to Re: Perl/Cgi : Suggestions on how to pass Large Hash Data to myscript.cgi? by Joost
in thread Perl/Cgi : Suggestions on how to pass Large Hash Data to myscript.cgi? by EchoAngel

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.