Perl: the Markov chain saw | |
PerlMonks |
Re: Processing ~1 Trillion recordsby dHarry (Abbot) |
on Oct 25, 2012 at 10:06 UTC ( [id://1000798]=note: print w/replies, xml ) | Need Help?? |
Can you give some background information of the problem at hand? What database do you use? What does the query look like? Restructuring a massive array and writing CSV files sounds costly. What pattern? How many CSV files? Profiling a (small) subset of the data might give some indication. Did you time how long the various steps take? In short: could you show some code please?
In Section
Seekers of Perl Wisdom
|
|