if I were to present Judy::HS to $work, as a buggy module, which needed patching, and appeared to be abandonware, it probably wouldn't be received too well

Agreed. I find the topic of third party dependencies a difficult and fascinating one -- see Re: Criteria for when to use a cpan module (Buy vs Build).

Early results do show that Judy::HS used a lot less memory than %hash

That agrees with marioroy's experiences and tye's experiences and BrowserUk's experiences:

A supersearch for my name and judy arrays will turn up a compact, single file version of the Judy array code that compiled clean and worked very well for that application. Still slower than hashes, but far more compact. Just hope you don't find any bugs, because the Judy code is horribly complex.

-- BrowserUk in Re: Fastest way to lookup a point in a set

Assuming you see big savings in memory (but not speed), would you recommend Judy at $work? That would seem to depend on your company's business model.

If your software must run on thousands of (many different) customer machines, then buying more memory is not an option.

OTOH, if you're running high performance code for clients on machines that you own and control, it may be better to stick with Perl hashes and just buy more memory. After all, a DDR4 DIMM can hold up to 64 GB while DDR5 octuples that to 512 GB, so I expect the cost of buying more memory for multiple in-house machines would be dwarfed by your monthly salary bill.

It's also a low risk solution because no code changes are required and you don't need to worry about Judy being abandoned or hitting a nasty bug in the notoriously complex Judy C code that you need to urgently fix. There's also the opportunity cost of your people working on Judy code instead of other projects.

All very interesting; there should be a Meditation somewhere down the track with results of this investigation.

Looking forward to it. :)

Update: In Re^3: 32bit/64bit hash function: Use perls internal hash function? (Apr 2022) hv reports that a proposed Nicholas Clark change to improve Perl hash performance in perl 5.38 reduced memory usage by 20% and gave substantial speed improvements in one of his test cases (perl 5.38 release date prediction market).

See also:

Update: Building Perl from Source References

In reply to Re^5: Size of Judy::HS array: where is MemUsed()? - perldelta, Perl Releases and Building Perl by eyepopslikeamosquito
in thread Size of Judy::HS array: where is MemUsed()? by kcott

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.