in reply to Re: Simple Digests?
in thread Simple Digests?

Thanks for the response!

Actually, I need to be able to use any data as the key. Since the data needs to be storable in standard DB file formats, I need a way to handle relatively long data. One possible example would be an entire email. It could also be, I dunno, a JPEG file...

Replies are listed 'Best First'.
Re^3: Simple Digests?
by radiantmatrix (Parson) on Mar 29, 2006 at 15:13 UTC

    Ah. That's a tough one. Digests can help, but since two very different sets of data can share a digest... well, it won't be unique. You could do something like a digest prepended with the nth 10 bytes from the file, or something, but even that wouldn't be a guarantee.

    It comes down to finding something that's unique enough about the data that when it's combined with a digest, you have a "pretty much guaranteed" unique key. MIME type, maybe? Combined with a longer digest (say, SHA-256), that would be pretty good.

    Two different digests (say, MD5().SHA256()) would be a likely candidate, too -- the chance that data "A" will have a digest collision with data "B" in two different digest systems is fanstastically small.

    <-radiant.matrix->
    A collection of thoughts and links from the minds of geeks
    The Code that can be seen is not the true Code
    I haven't found a problem yet that can't be solved by a well-placed trebuchet

      I really don't need that much protection from collisions. It's not like I need to worry about someone intentionally trying to create a collision. Instead of a 1 in a Zillion odds of a collision, I probably could get by with 1 in a thousand.

Re^3: Simple Digests?
by shotgunefx (Parson) on Mar 29, 2006 at 21:04 UTC
    Are you retrieving it just by the key? If you're using some other data to get the key, you could use a timestamp prefixed to the md5.
    Seems like it would be pretty tough to have a collision

    -Lee
    "To be civilized is to deny one's nature."

      Freaking Brilliant!

        Sometimes ;)

        -Lee
        "To be civilized is to deny one's nature."