Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

Re: Re: A memory efficient hash, trading off speed - does it already exist?

by JPaul (Hermit)
on Feb 04, 2003 at 04:37 UTC ( [id://232463]=note: print w/replies, xml ) Need Help??


in reply to Re: A memory efficient hash, trading off speed - does it already exist?
in thread A memory efficient hash, trading off speed - does it already exist?

Indeed I am also considering something similar to this, although I'm not entirely sure if it would work quite as I hope.

The data is constructed like so:

%hash{Keyword} -> {Symbol_1} = Weighting %hash{Keyword} -> {Symbol_2} = Weighting
And so on. I think perhaps I could slim down memory considerably by doing something like the following:
%hash{Keyword} = "Symbol_1 | Weighting, Symbol_2 | Weighting"
And, as Mr_Person has already noted, no I don't intend to use | and , as separators. Already when writing data to the disk as a "brainsave" I use \x255 as a record separator, just like you were thinking. Perhaps the fact that I started programming in BASIC as well might have something to do with our shared mindset? :)

JP,
-- Alexander Widdlemouse undid his bellybutton and his bum dropped off --

Replies are listed 'Best First'.
Re^3: A memory efficient hash, trading off speed - does it already exist?
by Aristotle (Chancellor) on Feb 06, 2003 at 11:57 UTC
    If anything, you should probably switch to a structure like
    $hash{"Keyword\0Symbol_1"} = $Weighting[1]; $hash{"Keyword\0Symbol_2"} = $Weighting[2];

    That way you still get full hash lookup performance on the entire (two dimensional) key. And you avoid creating whole legions of hashes. That should pretty much fix 90% of your problem. And such a flat hash is trivial to tie to a DBM file.

    Don't knock DBMs without benchmarking them. DBM files are specifically laid out to allow for rapid retrieval - paging just indiscriminately swaps out whatever seems least necessary, including other parts of the system besides the data. Drawing any conclusions about one based on the other borders on ridiculous.

    Makeshifts last the longest.

      This requires I know what the following symbol is.
      The point of the hash is to look up potential following symbols of the keyword, unless there's someway to glob hash keys without going through them to compile a list, it's not going to work at all.

      JP,
      -- Alexander Widdlemouse undid his bellybutton and his bum dropped off --

        Then keep a key with a list of symbols.
        $hash{"Keyword"} = "Symbol_1\0Symbol_2\0Symbol_3"; my $key = "Keyword"; for my $symbol (split /\0/, $hash{$key}, -1) { my $weight = $hash{"$key\0$symbol"}; # ... }

        Makeshifts last the longest.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://232463]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others romping around the Monastery: (6)
As of 2024-03-29 12:53 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found