Memory consumption with

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hello there, I'm doing my first tests with threads in perl. Using the worker-model my program starts several threads and detach them. Unfortunatly after the threads ended up, it seems that perl doesn't clean up the memory. The program uses as much memory as before. I tried a little test program:

use threads;

my $i=1;
while($i<=50){
    my $thr = threads->new(\&lwp);
    $thr->detach();
    print $i++, "\n";
    sleep(1);
};
sleep();

sub lwp {
    sleep(60);
    print threads->tid." ended\n";
    return 1;
}
[download]

After starting the 50 threads the program consumes approx 18 mb of memory, after the threads ended this is slightly reduced to 17 (running perl 5.8.4 on a linux machine). Anyone can help me with that?

Comment on Memory consumption with Download Code

Replies are listed 'Best First'.
Re: Memory consumption with by tirwhan (Abbot) on Nov 29, 2005 at 14:47 UTC
perl does not give memory back to the operating system (see `perldoc perlfaq3` for more on this), so the memory consumption of the process will not shrink during its execution time. It does however reuse memory which it has claimed and freed internally. So if you start another 50 threads after the first 50 have ended you should not see the memory consumption of your process go up significantly (provided you are not keeping a reference to an old data structure around somewhere). Update: as Fletch has said, using separate processes instead of threads will achieve the desired effect, because the operating system reclaims memory from processes that have ended. Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. -- Brian W. Kernighan	[reply] [d/l]
Re: Memory consumption with by Fletch (Bishop) on Nov 29, 2005 at 15:05 UTC
You might want to look at using child processes (via something like Parallel::ForkManager or the like) instead, as those will release their resources back to the OS when the child exits.	[reply]
Re: Memory consumption with by zentara (Cardinal) on Nov 29, 2005 at 16:48 UTC
You have to treat threads like objects, and reuse them if you want to reuse their memory. Detaching does not mean they are a separate process. What you could do, would be to create 5 threads, and NOT detach them, then run each thread 10 times, then join them. That way, your memory usage will increase to the size-of 5 threads, but will not increase on each thread run. Unless of course you do something different in a thread-run, like create a huge structure in memory. I'm not really a human, but I play one on earth. flash japh	[reply]