Re^2: Iterating through Two Arrays. Is there a better use of memory?

Again, many thanks. Any yes, everyone is correct. The numbers I gave won't effect my memory now, but I'm working on a preliminary model. Which means, I will probably run into some memory issues later. I know for the actual model I am dealing with at least two files that are 3.6GiB and 5.9GiB. Others could potentially be larger. I will heed and take everyone's advice!thanks!

Jeri

Comment on Re^2: Iterating through Two Arrays. Is there a better use of memory?

Replies are listed 'Best First'.
Re^3: Iterating through Two Arrays. Is there a better use of memory? by ikegami (Patriarch) on Oct 13, 2011 at 17:56 UTC
I know for the actual model I am dealing with at least two files that are 3.6GiB and 5.9GiB. Totally irrelevant to your question. The sum of the number of elements in `@line1` and `@line2` is what matters. You said the product of their size is going to be 10,000, so the sum of their sizes is going to be between 200 and 10,001, rather small numbers. If you can fit 10GiB into memory, don't fret about using less than 100KB to loop through the data.	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re^3: Iterating through Two Arrays. Is there a better use of memory?
by ikegami (Patriarch) on Oct 13, 2011 at 17:56 UTC

I know for the actual model I am dealing with at least two files that are 3.6GiB and 5.9GiB.

Totally irrelevant to your question. The sum of the number of elements in @line1 and @line2 is what matters. You said the product of their size is going to be 10,000, so the sum of their sizes is going to be between 200 and 10,001, rather small numbers.

If you can fit 10GiB into memory, don't fret about using less than 100KB to loop through the data.

[reply]
[d/l]
[select]