Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

comment on

( [id://3333]=superdoc: print w/replies, xml ) Need Help??

Take a look at Tie::SubstrHash. I think that it's a part of the standard distribution. It's somewhat slower and a bit clumbsy to use, but you do get to control the memory it uses. How much it will save you will depend on your application.

One tip. You have to specify size (in bytes) for the keys, the values and the total table size. This often isn't a problem for the keys but can make it awkward and less memory efficient than it might be if most of your values are short but you need to pre-allocate extra length to accomodate the occasional long value. You can work around this limitation by storing references to scalars instead of the scalar itself. This means you only need to allocate 4-byte per value, but that the real value can be any length. Of course it adds a dereference to the equation. The other benefit of this is that by using references, you don't have to pad the value before assigning.

You will have to pad the keys to the pre-specified length though. Quite why this isn't done for you is not clear to me. However, it was a simple change to make to my copy of the module. I also made it so that over-long keys are automatically truncated. Slightly dangerous, but provided you ensure that the pre-allocated size is sufficient for your keys not to clash, it greatly simplifies using the module.

Caveat implementor should you decide to make this change yourself.

If the module works for you, it should be an easy adaption to your present code.


Examine what is said, not who speaks.

The 7th Rule of perl club is -- pearl clubs are easily damaged. Use a diamond club instead.


In reply to Re: A memory efficient hash, trading off speed - does it already exist? by BrowserUk
in thread A memory efficient hash, trading off speed - does it already exist? by JPaul

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others lurking in the Monastery: (8)
As of 2024-03-28 17:07 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found