go ahead... be a heretic | |
PerlMonks |
Re: (tye)Re: Sorting data that don't fit in memoryby jeroenes (Priest) |
on Jan 24, 2001 at 20:07 UTC ( [id://54011]=note: print w/replies, xml ) | Need Help?? |
Read the note at the bottom
I'm impressed again. I was just installing BerkeleyDB according to tilly's suggestion. That was a real nice one, as well. I like using existing modules, as opposed to some others@pm. However, your code seems to be exactly what I wanted. Well, the size of the string $sorton shouldn't be much of a problem, it only uses (in my case) 2*13M = 26M of memory. Small offer, here. I did a check whether my local 'sort' (qsort, apparently) messed with the order of equal keys: So, it doesn't. I'm very happy with that. This, because the remainder of the string is some code for the time, stored in something like 100us accuray, with a max of 2-3 days. You don't want to sort on that, but you also don't want to change the order. I hope this explains my reluctancy to mix it up. I guess the key notion here is the collection of my keys in a string, saving the array-overhead. Thanks a lot, Jeroen Update I'm afraid I was too fast with my happiness. The real memory hog lies in the 0..($idx-1). That gives you 13M item array, that won't fit in memory. Too bad.
In Section
Seekers of Perl Wisdom
|
|