My problem with that is that if I write a threaded program that stays running (like a proxy server or a web server) the longer it runs the more memory it takes.

Unfortunately, there is still some memory used by the, (unnecessary in most cases!), cloning process that is performed when a thread is spawned, that Perl's memory management never recovers. This is the output from a modified version of your OP code that loops 50 times over the thread spawning/death measuring the memory at each stage:

C:\Perl\src\perl-5.8.8\win32>..\perl -mlib=\perl\site\lib \test\532177 +.pl StartMem: 3344 MB Pass 1 MidMem: 18340 MB EndMem: 5228 MB Pass 2 MidMem: 18380 MB EndMem: 5976 MB Pass 3 MidMem: 18384 MB EndMem: 6900 MB Pass 4 MidMem: 18396 MB EndMem: 6896 MB Pass 5 MidMem: 18396 MB EndMem: 7356 MB Pass 6 MidMem: 18392 MB EndMem: 7796 MB Pass 7 MidMem: 18392 MB EndMem: 7984 MB Pass 8 MidMem: 18392 MB EndMem: 7928 MB Pass 9 MidMem: 18392 MB EndMem: 8064 MB Pass 10 MidMem: 18400 MB EndMem: 7968 MB Pass 11 MidMem: 18400 MB EndMem: 8928 MB Pass 12 MidMem: 18400 MB EndMem: 8544 MB Pass 13 MidMem: 18400 MB EndMem: 8612 MB Pass 14 MidMem: 18400 MB EndMem: 8736 MB Pass 15 MidMem: 18400 MB EndMem: 8620 MB Pass 16 MidMem: 18400 MB EndMem: 8540 MB Pass 17 MidMem: 18400 MB EndMem: 8256 MB Pass 18 MidMem: 18404 MB EndMem: 8968 MB Pass 19 MidMem: 18404 MB EndMem: 8652 MB Pass 20 MidMem: 18404 MB EndMem: 8580 MB Pass 21 MidMem: 18396 MB EndMem: 9028 MB Pass 22 MidMem: 18408 MB EndMem: 8852 MB Pass 23 MidMem: 18404 MB EndMem: 9272 MB Pass 24 MidMem: 18404 MB EndMem: 8812 MB Pass 25 MidMem: 18404 MB EndMem: 9012 MB Pass 26 MidMem: 18404 MB EndMem: 8980 MB Pass 27 MidMem: 18412 MB EndMem: 9480 MB Pass 28 MidMem: 18408 MB EndMem: 9524 MB Pass 29 MidMem: 18400 MB EndMem: 9352 MB Pass 30 MidMem: 18408 MB EndMem: 9508 MB Pass 31 MidMem: 18404 MB EndMem: 9240 MB Pass 32 MidMem: 18404 MB EndMem: 9084 MB Pass 33 MidMem: 18404 MB EndMem: 9324 MB Pass 34 MidMem: 18412 MB EndMem: 9368 MB Pass 35 MidMem: 18416 MB EndMem: 9240 MB Pass 36 MidMem: 18420 MB EndMem: 9500 MB Pass 37 MidMem: 18412 MB EndMem: 9532 MB Pass 38 MidMem: 18412 MB EndMem: 9680 MB Pass 39 MidMem: 18424 MB EndMem: 9596 MB Pass 40 MidMem: 18424 MB EndMem: 10116 MB Pass 41 MidMem: 18424 MB EndMem: 10256 MB Pass 42 MidMem: 18416 MB EndMem: 10092 MB Pass 43 MidMem: 18428 MB EndMem: 10028 MB Pass 44 MidMem: 18428 MB EndMem: 10060 MB Pass 45 MidMem: 18428 MB EndMem: 10388 MB Pass 46 MidMem: 18420 MB EndMem: 10248 MB Pass 47 MidMem: 18428 MB EndMem: 10348 MB Pass 48 MidMem: 18420 MB EndMem: 10320 MB Pass 49 MidMem: 18428 MB EndMem: 10084 MB Pass 50 MidMem: 18420 MB EndMem: 10540 MB

As you can see, whilst the vast majority of the memory consumed by the spawned threads is recovered after each pass, there is a residual growth. And even though the residual sometimes falls back further on one pass than it did the previous pass, there is an inexorable upwards trend that cannot be explained by heap growth or memory fragmentation. I think that does consitute a leak.

This is one reason why I continue to advocate using a few long-running threads rather than spawning threads that "do & die". I prefer long running threads that wait on a queue for work, do the work, then go back to get another batch to do anyway, but given the continued leaks during the cloning process, it seems to be the only way to get long running threaded Perl processes to work properly.

Depending upon what you are doing and how your code is currently structured, it can be quite easy to convert "do & die" code to the "wait/run/wait" model, and it generally speaking tends to be more efficient anyway. I'd be willing to try and help you make the transition.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

In reply to Re^3: Cleaning up threads so they do not leak memory. (Win32) by BrowserUk
in thread Cleaning up threads so they do not leak memory. (Win32) by slloyd

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.