in reply to Optimizing Memory consumption
This isn't answering the question you asked; Mostly because I'm not convinced you are asking the right question.
My Rule of Thumb is that anytime there is a significant change to requirements, there needs to be a significant review of the architecture. I would say that a quadrupling of your memory requirements and the fact that you are using a 200MB text file confirms this.
There isn't really enough info about your situation to come to any realistic conclusion because you have already decided to optimize (fair enough). However, when I see a monstrous text file, I always wonder if the use of a Relational DB and clever queries would generate a more efficient solution. You mention that "the data to be stored has increased by (a?) large (margin?)" this is often a point at which data abstraction needs to be reviewed.
Finally, please forgive me if I am off base.
(update: recomposed 1 sentence.)
Love,
Brig
|
|---|