|There's more than one way to do things|
I've just run a (not yet thoroughly tested; nor cleaned up; nor even fully optimised) solution to my problem, and the results are promising:
For context, here is the single Perl hash solution from the other thread on the same dataset:
That equates to just 7% longer to run the same dataset using just over 1/5th the memory. Not bad for a fairly crude POC with still considerable potential for further time reduction.
So, I've expended 3 days -- call it ~50 hrs of my time -- to optimise this.
No matter how you look at it, that is a far better trade-off than your criminally inept suggestion -- to move to a disk-based DB -- which at (a minimum) of 5000x slower, would cost my customer $2800 * 5000 = $14 million for every hour of processing he intends to do.
And given the proposal is to run 100 hours of simulations; the net cost of your suggestion would be $1.4 billion - $280,000 =
I just hope that people will learn from this and finally realise that your malodorously constructed, maliciously provocative, willfully ignorant, hopelessly outdated -- not just non-useful, but perilously inept -- completely worthless, merit-less fluff; is intended to be neither technically useful nor intellectually stimulating; its only purpose being shameful self-promotion of the vilest, any-hits-are-good-hits, form.
With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
In reply to Re^2: RAM: It isn't free . . .(Mike Robinson:criminally inept.)