Thanks for the suggestions I got so far. I'm doing this in memory
to clean up the data (nice and fast) before dumping it to a
database.
Here are some timing results for a real example with
$i = 0..21448 and $j 0..91 on an Athlon 1.2GHz with 1GB RAM.
Timing is for creating the array (about 5 sec) and the rest of the
time goes into garbage collection. Total size of this
process is about 112 MB, so about 10% of machine RAM. CPU
usage is about 99+ percent and no swapping. (This is a small
example that I can test in finite time :-( )
undef @data;
This takes 2 min 56 sec
Now do
for (my $i=0;$i<@data;$i++)
{
undef @{$data[$i]);
}
This takes 1 min 39 sec
Finally do
for (my $i=0;$i<@data;$i++)
{
$data[$i] = ();
}
This takes 43 seconds.
Adding
@data =() to the end of the last
example really makes no difference performance wise.