in reply to Re: Re: Re: character-by-character in a huge file
in thread character-by-character in a huge file

Three questions:

  1. Your files are in FASTA format?
  2. Is there a maximum size for individual records?
  3. When processing the records byte by byte, how do you intend treating the inter-record newlines?

Examine what is said, not who speaks.
"Efficiency is intelligent laziness." -David Dunham
"Think for yourself!" - Abigail
  • Comment on Re: Re: Re: Re: character-by-character in a huge file

Replies are listed 'Best First'.
Re: character-by-character in a huge file
by mushnik (Acolyte) on Apr 13, 2004 at 17:14 UTC
    "Your files are in FASTA format?"
    Yes.

    "Is there a maximum size for individual records?"
    A few MB. I can't be more specific than that, because I haven't seen all the files in the world...but generally, the largest scaffolds I've seen are a coupe MB.

    "When processing the records byte by byte, how do you intend treating the inter-record newlines?"
    Reasonable question. I just skip them. This is handled in a function call to "get_next_char", which really gets the next char about which I care. Also, see a ">", skip until the newline.