Hi,
I got in some trouble with the memory consumption of a threaded tcp server.
I created a new thread for every connect, and destroyed the thread after the connection has been closed.
However, I got an "Out of memory" after creating and destroying a few 100 threads, so I tried to figure out what's wrong.

I wrote a testscript, which constantly creates and exits threads, it's below.
This testscript will also run out of memory, but the memory consumption is quite unpredictable and seems to raise in steps.
Changing the sleeping time below the comment # SLEEP does have some effect, as longer the sleep time is here as slower the memory consumption raises.

Is there some way to avoid the memory leak ?

Thanks, Michael
#!/usr/bin/perl -w

use threads;
use threads::shared;

share $threadscount;
$threadscount = 0;
share $maxthreads;
$maxthreads = 0;
share $threadscreated;
$threadscreated = 0;

# Dump out virtual memory and resident memory consumption
sub printmemorysizes{
        open F, "</proc/self/status";
        my @s = <F>;
        close F;
        my @vmsize = grep /^VmSize.*/, @s;
        chomp $vmsize[0];
        print $vmsize[0], " ";
        my @rss = grep /^VmRSS/, @s;
        print $rss[0];
}

# The treads' sub
sub thread{
        my $count;
        { 
                lock $threadscount;
                $threadscount ++; # count how many threads there are
                $count = $threadscount;

                lock $maxthreads;
                if ( $count>$maxthreads ){
                        $maxthreads = $count; 
   # Store the maximum of concurrent threads
                }
                lock $threadscreated;
                $threadscreated ++;
        }
        #print "threadscreated: $threadscreated ";
        #&printmemorysizes();
        select undef,undef,undef,1;

        lock $threadscount;
        $threadscount --;
}

while ( 1 ){
        for ( 1..300 ){
                my $t = threads->create("thread");
                $t->detach();
        }
        my $tc;
        do {
                select undef,undef,undef,0.1; # SLEEP
# as shorter the sleep the memory cunsumption raises faster it seems
                
                lock $threadscount;
                $tc = $threadscount;
        } while ( $tc );
        #sleep 2;

# dump out: 
#maximum concurrent threads | How many threads have been created | 
# virtual memory size | resident memory size
print "maxthreads: $maxthreads  threadscreated: $threadscreated  ";
        printmemorysizes();
        $maxthreads = 0;

}

[download]
------------- Some example output (nothing changed between the runs) 
micha@laptop ~/prog/perl/test $ ./threads_leak.pl
maxthreads: 300  threadscreated: 300  VmSize:     177468 kB VmRSS:    
+    125216 kB
maxthreads: 300  threadscreated: 600  VmSize:     291888 kB VmRSS:    
+    248072 kB
maxthreads: 300  threadscreated: 900  VmSize:     326356 kB VmRSS:    
+    248520 kB
maxthreads: 300  threadscreated: 1200  VmSize:    426424 kB VmRSS:    
+    249172 kB
maxthreads: 300  threadscreated: 1500  VmSize:    287920 kB VmRSS:    
+    252020 kB
maxthreads: 300  threadscreated: 1800  VmSize:    287920 kB VmRSS:    
+    252464 kB
maxthreads: 300  threadscreated: 2100  VmSize:    313928 kB VmRSS:    
+    252736 kB
maxthreads: 300  threadscreated: 2400  VmSize:   1014284 kB VmRSS:    
+    253800 kB
maxthreads: 300  threadscreated: 2700  VmSize:    525608 kB VmRSS:    
+    372676 kB

micha@laptop ~/prog/perl/test $ ./threads_leak.pl
maxthreads: 300  threadscreated: 300  VmSize:     161832 kB VmRSS:    
+    125272 kB
maxthreads: 300  threadscreated: 600  VmSize:    1873224 kB VmRSS:    
+    128360 kB
maxthreads: 300  threadscreated: 900  VmSize:     288048 kB VmRSS:    
+    248756 kB
maxthreads: 300  threadscreated: 1200  VmSize:   1367664 kB VmRSS:    
+    250560 kB
maxthreads: 300  threadscreated: 1500  VmSize:    370560 kB VmRSS:    
+    249440 kB
maxthreads: 300  threadscreated: 1800  VmSize:    724704 kB VmRSS:    
+    250108 kB
maxthreads: 300  threadscreated: 2100  VmSize:    410060 kB VmRSS:    
+    372136 kB

[download]

In reply to memory leaks with threads by misc

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.