mohangopa:

I've found that hashes work even when the datasets are fairly large, especially if the hash entries themselves aren't too big. I'd go ahead and try it and see how well hashes work in your application.

Once you have it running, give it a file twice the normal size, and watch the memory it consumes as it runs. If all goes well, then I'd guess you're done until your processing volume doubles. If it had RAM trouble, then I'd suggest using something like SQLite or DBM to give you quick local cache with disk backup. That way, you can handle larger data structures without the overhead of communicating with the external database.

...roboticus

When your only tool is a hammer, all problems look like your thumb.

In reply to Re^3: Perl Caching by roboticus
in thread Perl Caching by mohangopa

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.