in reply to Re^3: storing hash in temporary files to save memory usage
in thread storing hash in temporary files to save memory usage

All dna sequences are of equal length but 150 characters long. may be sometimes we have data of more characters but all will be of same length.

yes they consist of combinations of AGTCN

  • Comment on Re^4: storing hash in temporary files to save memory usage

Replies are listed 'Best First'.
Re^5: storing hash in temporary files to save memory usage
by BrowserUk (Patriarch) on Sep 21, 2016 at 06:47 UTC

    The indicated lines below are your keys for matching:

    data1.txt
@NS500278
AGATCNGAA                           ********
+
=CCGGGCGG   1

@NS500278
TACAGNGAG                           ********
+
CCCGGGGGG   2

    [download]

    but they are 150 chars long? (Even though you've shown them as 9!?)

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]

      yes they are 150 characters long but for simplicity of the example and for others to understand I have shown them as 9 but the length should not be fixed because if I try my code with some other data it should also work then.

[reply]

        Please take a look at some of the (many) posts by BrowserUk around this place that deal with Big Data processing, then carefully consider if you may not want to trim your requirements a bit.

        Give a man a fish:  <%-{-{-{-<

[reply]
[d/l]
        but the length should not be fixed because if I try my code with some other data it should also work then.

        Oh. Okay. Good luck with finding your fully generic solution. (Why not see if it can cure world hunger at the same time.)

        When you find the other solutions taking days or weeks and you are prepared to lower your expectation, come on back.

        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
        In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

In Section Seekers of Perl Wisdom