Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Re: A memory efficient hash, trading off speed - does it already exist?

by toma (Vicar)
on Feb 04, 2003 at 06:26 UTC ( [id://232476]=note: print w/replies, xml ) Need Help??


in reply to A memory efficient hash, trading off speed - does it already exist?

Check out the soon-to-be-renamed Table-ParentChild by CPAN author MIKEWONG. It will store the type of relationship that you need in a more compact way. To use it, you would create an index for each string to be stored, and then store the relationships using a pair of indices.

The 120,000 combinations should take about 16 bytes each, resulting in under 2MB of storage. Additional memory will be required for the word<->index lookup data structure.

This code is fast, but not quite as fast as a hash. It uses much less memory. It works by using a linked list in C, which is written as an XS module.

If it looks interesting, you could ask Mike to upload his latest version to CPAN. In the existing version, the relationship is stored as an integer. In the newer version, the relationship can optionally be a perl scalar. If you turn on the scalar option, it uses more more memory, of course.

Update:Based on measurements from MarkM it looks like I may be optimistic by a factor of two (or more, depending on your data) in my memory estimates.

It should work perfectly the first time! - toma

  • Comment on Re: A memory efficient hash, trading off speed - does it already exist?

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://232476]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others examining the Monastery: (7)
As of 2024-04-19 08:37 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found