I thought that PERL should be doing garbage collection once the threads terminate.
It does. But -- unlike fork processes -- that memory is returned to the process free memory pool, not to the operating system.
That is, the memory used by a ended thread is freed, and can be reused for the next thread -- that is why when you only use one thread at a time, the memory used by the process stays steady. The memory allocated from the OS for the first thread; can be reused when you start the second thread.
When you run more than one thread concurrently, the process will obviously need more memory; but as old threads end and new ones start; the memory will be recycled for those new threads.
In other words; you aren't seeing a memory leak; just memory usage, so stop worrying about it.
Now, if you are running out of memory, then you are either: a) not terminating old threads cleanly; b) or using more threads than you have memory to support.
If the former; there is usually a fairly obvious reason why threads fail to end cleanly; but I'd need to see the real code.
If the latter; it is usually a design flaw. Most modern systems (4GB min) can support upwards of 100 concurrently threads -- with care, I have had over 3000 concurrent threads running on my 8GB system -- but just because you can have a large number of threads concurrently running; it rarely if ever makes sense to do so. And a design that requires it is usually the wrong design.
Sometimes running out of memory with low(ish) numbers of threads indicates that you are inadvertently duplicating large chunks of memory that you did not intend to.
And finally, there are steps you can take to reduce the memory usage of your threads by careful consideration of what and when you load modules. The default habit of useing modules at the top of the program means that they all get replicated into every thread; regardless of whether those threads need them or not. It's convenient; but can cause excessive memory use.
The bottom line is; if you have a genuine problem with memory usage of your threaded code; and you want to address it; you will have to post the real code, not snippets.
With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
|