http://qs1969.pair.com?node_id=641586


in reply to Re: Strings and numbers: losing memory and mind. (SV sizes)
in thread Strings and numbers: losing memory and mind.

Your tactic strikes me as something that is usually a waste to worry about before actually determining that it matters in the particular situation.

I agree! Perhaps I should have prefaced my meditation by saying that this would not have been a problem I'd have needed to solve if the data set were not so large. As it was, reading 50 million strings took a little less than 5G of memory, and then it started eating up more during processing. Using 50 million ints instead took only about 2G of memory (and, of course, didn't grow). It's the difference between "just fits" and "won't work."

Tracking this down was such a puzzle for me because I've really never had to worry about it before. Strings and numbers frolic freely together. Perl worries about the details, and I don't.