Actually you may have a point regarding the key. However im not so sure about the ondisk versus in ram issue. Im on a box with large ram, and virtually all of the complex data structures will be reused several times, so my thinking is that I will just let the OS decide if the data should be swapped out or not. Only one of the large strucutures is used at a time, and is used for a reasonable amount of time, so I suspect that avoiding swapping isnt a win at all, and possibly even would be a loss. BTW, the 8000x2 data elements end up being about 100k+ of RAM, but even there its more the construction time than the memory that im worried about. It takes a while to build that 100k structure. (Ive actually contemplated rewriting Tie::Array::PackedC in C, or recoding not to use it at all to reduce the run time, but at this point im overall satisfied with the performance.)

Anyway, more brainfood to digest. Thanks.


---
demerphq

    First they ignore you, then they laugh at you, then they fight you, then you win.
    -- Gandhi

In reply to Re^2: Bitten by the worst case (or why it pays to know whats inside the black box) by demerphq
in thread Bitten by the worst case (or why it pays to know whats inside the black box) by demerphq

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.