Re^4: Massive Perl Memory Leak

%datahash is initialized as nul since it's not used until we're already inside the forever loop. So when local reinstantiates it at each pass it "restores" it's value to that of the nul list, (). In my usage, I could replace it with %datahash = (); and the functionality would be identical. %datahash is also an unshared variable. No other threads can access it. So so long as Perl's innards are clean, no other thread can tamper with %datahash. I'm using the term thread global to mean it's global only within that one thread. To get program globals u have to share() them.

Does anybody know of a way to "see" data/memory that doesn't have a varname pointing to it? To find out what "anonymous storage" is in use.

Comment on Re^4: Massive Perl Memory Leak

Replies are listed 'Best First'.
Re^5: Massive Perl Memory Leak by BrowserUk (Patriarch) on Jun 11, 2007 at 19:50 UTC
I think you are barking up the wrong tree looking at %datahash as the cause of the problem. The following simple script emulated what you are describing and once the 101 threads are up and running, the memory usage is rock steady under both 5.8.6/AS811 with the AS delivered core version of threads and 5.8.8/AS817 with the latest cpan versions of threads & threads::shared. And that's relying upon Perl's garbage collection alone. More information needed. (I still suspect Data::Dump) #! perl -slw use threads; use threads::shared; print $threads::VERSION; print $threads::shared::VERSION; print $]; @th = map{ threads->new( sub{ while( 1 ){ local %h = ( 1 .. 10000 ); sleep 1 } } ) } 1 .. 100; $_->join for @th __END__ c:\test>junk9 Name "main::h" used only once: possible typo at c:\test\junk9.pl line +13. 1.05 0.92 5.008006 Terminating on signal SIGINT(2) c:\test>\AS817\perl\bin\perl5.8.8.exe junk9.pl Name "main::h" used only once: possible typo at junk9.pl line 13. 1.1201 0.99 5.008008 Terminating on signal SIGINT(2) [download] Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply] [d/l]
Re^6: Massive Perl Memory Leak by wagnerc (Sexton) on Jun 13, 2007 at 20:08 UTC
Well I just restored %datahash usage to the rebuild of the script and it didn't leak. :) Kind of glad that's not it. I also just restored Data::Dump usage to the rebuild and am testing it now. We'll see if it goes down. I don't remember if I was using Data::Dump in the first place when I noticed the leak. Actually, yes. I remember now, I need Data::Dump to serialize the hash to pass it through a message queue to the database thread.	[reply]