Re^7: Massive Perl Memory Leak

If it were single threaded, yes. But remember threading is still Wild West territory

That's a total red-herring. Unless you are using shared variables (and even then), there is no effective difference in running two Perl threads and running two perl processes. Each thread has its own interpreter, just as each process does. And variables allocated by each interpreter are exclusive to that interpreter.

When variables die Perl only guarantees that the varname is no longer accessible, not that anything in particular has been done with the SV's behind the scenes.

That's wrong also. Reference counting means that variables allocated at any given scope are returned to the memory pool as soon as you leave that scope(*). With most other GC mechanism, those variables would be sitting around gathering electronic dust, inassessible but unreclaimed, until low memory or some other extraordinary event causes the whole program to freeze while the garbage collector scans all the programs dataspace, heap and stack,(twice at least), checking to see what is lying around and if anything else is still referencing it.

Which makes:

In fact the whole Perl memory management philosophy seems too lacidasical.

Just about the opposite of reality. Perl's GC mechanism, reference counting, is the most eager GC mechanism possible.

The only time that falls down is if you are creating circular references--as I mentioned above.

*Unless your code has passed a reference to the variables out of that scope and they have failed to let go of those references. That is, your code has done something that necessitates retention. There are a few special exceptions to do with optimisations for function lexicals also, but if they were the root of your problem, they would be known about by now.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

Comment on Re^7: Massive Perl Memory Leak

Replies are listed 'Best First'.
Re^8: Massive Perl Memory Leak by wagnerc (Sexton) on Jun 13, 2007 at 21:09 UTC
*Unless your code has passed a reference to the variables out of that scope and they have failed to let go of those references. That is, your code has done something that necessitates retention. There are a few special exceptions to do with optimisations for function lexicals also, but if they were the root of your problem, they would be known about by now. That's what I'm talking about. When varname goes out of scope, the name goes away and the refcount to the SV goes down by one. The problem I was refering to was obscure varnames/references that u don't know about holding the door open to the SV, thus preventing garbage collection. My second concern was when/where/how/if garbage collection takes place at all. It would be nice if there were a debugging feature where Perl would tell u when it gc'd something.	[reply]
Re^9: Massive Perl Memory Leak by BrowserUk (Patriarch) on Jun 14, 2007 at 01:18 UTC
The problem I was refering to was obscure varnames/references that u don't know about holding the door open to the SV, thus preventing garbage collection. But if you pass a reference to a locally scoped lexical out of that scope, then wherever you passed it to, may try to access that variable via the reference. In other words, you don't want it to disappear until that reference has been disposed of. Once the variable holding that reference goes out of its scope, then it will be GC'd. And a part of the process of GC'ing a reference, is to decrement the ref count of the thing it points at. And if doing that decrement drops the count to zero, then the now-out-of scope original lexical will also be GC'd. This mechanism works and has worked for a long time. Unless you stick a copy of the reference in a global variable and then forget about it, the reference will go out of scope, and the referent will be GC'd exactly when that happens. My second concern was when/where/how/if garbage collection takes place at all. It would be nice if there were a debugging feature where Perl would tell u when it gc'd something. There is. If you want to get into building your own perl and build a debug version, then there are flags that can be set to cause all memory allocations and frees to be traced. I don't know how to do it--I've never felt the need--but it's probably on a case of a make file/configure setting, and some environment variables. Followed by wading through a megasized trace file staring at a gob load of 8 digit hex values. Good luck with that. But if you'd posted the code 3 days ago as asked, your problem would probably be fixed by now. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply]