in reply to Re: Memory Efficient Alternatives to Hash of Array
in thread Memory Efficient Alternatives to Hash of Array

FYI Perl stops processing when it sees __DATA__ so there would be no problem loading a script that is over 4 GB of size. As for the use of a hash of arrays, reading the post I would assume a badly chosen data sample rather than a misunderstanding.

Update: Good catch, eye. The sample ws well chosen.

  • Comment on Re^2: Memory Efficient Alternatives to Hash of Array

Replies are listed 'Best First'.
Re^3: Memory Efficient Alternatives to Hash of Array
by eye (Chaplain) on Dec 27, 2008 at 20:19 UTC
    ...I would assume a badly chosen data sample...
    Actually, the OP's example has three sets of duplicate tags:
    Lines 6 - 9: TGATACGGCGACCACCGAGATCTACACTCTTTCC Lines 15 - 17: TGCTCCGGCGACCACCGAGATCTACACTCTTTCC Lines 19 - 20: TTCTCCTTCGACCACCGAGATCTACACTCTTTCC
Re^3: Memory Efficient Alternatives to Hash of Array
by BrowserUk (Patriarch) on Dec 27, 2008 at 20:43 UTC
    As for the use of a hash of arrays, reading the post I would assume a badly chosen data sample rather than a misunderstanding.

    Given the OPs description of the code: "My code below, tries to group the 'error_rate' (second column of data) based on its corresponding tag (first column of data).", in conjunction with that the second column appears to be a byte-wise mask for the first:

    AATACGGCCACCCCCCCCCCCCCCGCCCCTCCCC INILILFIIIIQNQQNQNLLKFKNCDHA?DAHHH

    I don't think it is just badly chosen sample data. Maybe the OP will tell us which is correct?


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      From the OP's text and code I thought that the OP wanted to know all of the possible values for the second field for each possible value of the first field. Given that the first field repeats, this requires an array.
        You are exactly right, tilly.

        ---
        neversaint and everlastingly indebted.......