What's happening is that perl's allocated lots of little chunks of memory. When perl exits, it needs to go through and hand them all back to the system like a well-behaved potentially embedded program should. Unfortunately the afflicted versions of libc, when presented with a zillion frees, seem to go insane managing its free list, with the performance problems you've noted. (This can also happen if you free up a lot of little chunks of memory in the middle of your program, in which case you'll get a noticeable pause)

-- Dan

Here strange for you. You don't mention what OS your using, though the "hitting ^Z and then killing the job" bit indicates some form of unix, but I tried a very unscientific experiment on my NT system and replicated your hash structure. It took between 12 and 15 seconds for Perl/the OS to release the memory (no swapping involved) if I used exit, but less than 4 seconds if I just allowed the script to fall off the end.

I had the test program print the time just prior to the exit/end-of-script and set my command prompt to display the time so I could see how long the process took.

I have no idea why calling exit would take so much longer than not, but maybe it would work for you too?

use Data::Dumper;

use strict;

use constant MAX_TOUCH => 30;
use constant MAX_EXP => 30;
use constant MAX_ROW => 20000;

my $hash;
my $exp;
my $row;
for ($exp = 0; $exp < MAX_EXP; $exp ++) {
    print "exp = $exp\n"; 
    $hash->{$exp} = {};
    for ($row = 0; $row < MAX_ROW; $row ++) {
        my $array = [1, 2, 3, 4, 5, 6, 7, 8, 9, 0];
        $hash->{$exp}->{$row} = $array;
    }
}

my $test;
for (0 .. MAX_TOUCH) {
    print "touch = $_\n"; 
    for ($exp = 0; $exp < MAX_EXP; $exp ++) {
        for ($row = 0; $row < MAX_ROW; $row ++) {
            $hash->{$exp}->{$row}[0] ++;
        }
    }
}

#print Dumper $hash;

print "about to exit\n";
exit;
[download]

I went throught it with the debugger, and tried exiting at different points, and the delay only occurred after the data was in memory. Also, in the debugger, I run  $data = undef, and that gave me the delay too.

And, yes, I do need that data in memory. I have to perform various manipulations and calculate statistics on the data which means I can't just process it one row at a time.

rdfield