in reply to Re: A better (problem-specific) perl hash?
in thread A better (problem-specific) perl hash?

What he said!

Convert each gene string to a number.

Depending on how much data you have, that isn't a hash.

It's a (sparse) array.

As long as your string length isn't too long that gives you a fixed cost storage, for as much data as you like (as long as it doesn't overflow the array element size)
  • Comment on Re^2: A better (problem-specific) perl hash?

Replies are listed 'Best First'.
Re^3: A better (problem-specific) perl hash?
by BrowserUk (Patriarch) on Feb 18, 2006 at 16:23 UTC
    (as long as it doesn't overflow the array element size)

    With 16 meg combinations of 12 bps, and a 3 billion (-11 :) bps in all, it averages to 179 of each 12-char strings (assuming random data). I used a 4 bytes integer for each counter, so even if the sequence consisted of entirely one character, there is still plenty of headroom.

    From some previous analysis I did on the Drosophila genome, the numbers of repeat sequences are quite low. In it's 160MB, repeats counts for subsequences of length 8 are in the low thousands and by the time you get to 12-chars rarely above a few hundred though you do get some 12-char combinations that run to several thousand repeats, particularly those involving T & A for some reason?

    I coded a version of the above in XS that allows you to specify the element size as 2, 4 or 8 and it will detect attempts to assign values larger. I'm having trouble with conversion between __int64s and doubles for returning to Perl though.


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
Re^3: A better (problem-specific) perl hash?
by BrowserUk (Patriarch) on Feb 18, 2006 at 17:31 UTC
    that isn't a hash, It's a (sparse) array.

    Actually, it's a Perfect Hash.


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.