a) Is there a better way to do this?
b) Can the memory footprint be reduced any?
Two very different questions. To answer the second question first, an easy way to reduce the memory footprint is for each query, read in the file line by line, and report if it overlaps. I bet you are now saying "but that's too slow". With many problems, there's a trade-off between memory usage, and processing time. Reduce the memory usage, and the processing time goes up. Just asking for "reduce the memory usage" without saying anything about processing time may not get the answer you are looking for.

As for you first question, it depends. What is "better" in your opinion? *My* opinion of better is to reduce query time, and invest in memory and preprocessing time - build a segment or interval tree and do queries against that. But that will increase your memory usage, so it's probably not better for you.

In reply to Re: Reducing memory footprint when doing a lookup of millions of coordinates by JavaFan
in thread Reducing memory footprint when doing a lookup of millions of coordinates by richardwfrancis

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.