in reply to Re^4: string to more compact format
in thread string to more compact format

Hmm strange that storing a 50 byte key hash is just a fraction larger than a 12-byte key hash. Why is that?

Because for each key/value pair, there are 40 bytes (32-bit, more on 64-bit) of overhead in addition to the key and value data. See Hash structure illustration. So for short keys, most of the space used by a hash is in the internal construction, not the keys & values themselves.

I can't store my strings into an array. Since sometimes strings occur more than once and i should keep track of that.

Then use the hash, but don't bother with the compression because you won't gain anything from it. 70MB isn't such a lot these days.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
RIP an inspiration; A true Folk's Guy