It sounds like what you want to is encode some long string
into something a LOT shorter representation that doesn't have to be unique.
This is a hash function.
There are all sorts of hash functions, but since we are programming in Perl, my initial thought would be: how does Perl do this?
This is the Perl hash function written in C:
int i = klen; unsigned int hash = 0; char *s = key; while (i--) hash = hash * 33 + *s++;
Internally Perl chops down the number of bits to get a practical index number by: index= hash & xhv_max. In your case, forget it - don't worry about what xhv_max is, use all the bits! (well maybe you should consider implications of the sign bit)klen is the number of characters to be encoded. 12345678901234567890123456789012345678901234567 the string above has 47 chars and would be klen for example.
If 32 bits isn't enough, then 64 bits is gonna do it, that's 8 bytes. Don't mess with 48 bits or something like that. Powers of 2 are magic on most machines commonly in use, eg: 2,4,8,16,32,64,128.
In summary, forget about SHA-1 or SHA-2 or any other form of encryption, use an efficient hash encoding technique. I would try a 32 or 64 bit version of what Perl itself does!
Update:The Perl hash algorithm works very well based upon my subjective empirical judgment with just 120K hash key structures. Anyway, I am confident that 20 bytes aren't necessary and that 8 bytes will yield the "uniqueness" that you need. That would allow the keys to be resident in memory. But, I think that the idea of using a DB is even better as it scales gracefully to HUGE structures.
In reply to Re^3: Question: methods to transfer a long hexadicimal into shorter string
by Marshall
in thread Question: methods to transfer a long hexadicimal into shorter string
by lihao
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |