Check out the soon-to-be-renamed Table-ParentChild by CPAN author MIKEWONG. It will store the type of relationship that you need in a more compact way. To use it, you would create an index for each string to be stored, and then store the relationships using a pair of indices.

The 120,000 combinations should take about 16 bytes each, resulting in under 2MB of storage. Additional memory will be required for the word<->index lookup data structure.

This code is fast, but not quite as fast as a hash. It uses much less memory. It works by using a linked list in C, which is written as an XS module.

If it looks interesting, you could ask Mike to upload his latest version to CPAN. In the existing version, the relationship is stored as an integer. In the newer version, the relationship can optionally be a perl scalar. If you turn on the scalar option, it uses more more memory, of course.

Update:Based on measurements from MarkM it looks like I may be optimistic by a factor of two (or more, depending on your data) in my memory estimates.

It should work perfectly the first time! - toma

In reply to Re: A memory efficient hash, trading off speed - does it already exist? by toma
in thread A memory efficient hash, trading off speed - does it already exist? by JPaul

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.