Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

comment on

( [id://3333]=superdoc: print w/replies, xml ) Need Help??

Good question.

My proposal is a single hash only too though. You can store the keyword -> symbollist and the keyword+symbol -> weighting entries in the same hash, provided your keywords never have a \0, but in that case the keyword+symbol or symol+weight packing will break as well anyway.

Obviously though, if you split the symbol string anyway, interleaving the weights into the same string comes at very little extra cost. In that case, my proposal doesn't really have a lot of merrit.

It will shine when you frequently know the symbol you're looking for beforhand and can thus skip the splitting step - then you can look up the weight directly. Looking up the same symbols frequently actually would be an argument in favour of this approach, as its key/value pairs constitute a much smaller amount of data each, allowing the OS and/or DBM drives to keep a lot more individual entries in their respective caches. It will still take up considerably more total memory than your approach (infrastructure for one extra key per symbol, plus the size of an extra copy of the keyword and symbol to store as the key) - but in the case of infrequent accesses to the symbol list key it actually works in favour of caching.

Another option to consider if you need to discover possible values might be BerkeleyDB's B-tree mode, which lets you store multiple values with identical keys and then query them in sequence. It's usually more expensive and less efficient than hash mode though, so you might do well to do something like use 53 DBMs keyed on the first character of the keyword (one for each letter, case sensitive, and one for non-letter characters) to split up the expense among individually more manageable data sets.

Makeshifts last the longest.


In reply to Re^7: A memory efficient hash, trading off speed - does it already exist? by Aristotle
in thread A memory efficient hash, trading off speed - does it already exist? by JPaul

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others chilling in the Monastery: (3)
As of 2024-04-16 22:59 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found