in reply to Re^4: Massive Perl Memory Leak
in thread Massive Perl Memory Leak

I think you are barking up the wrong tree looking at %datahash as the cause of the problem.

The following simple script emulated what you are describing and once the 101 threads are up and running, the memory usage is rock steady under both 5.8.6/AS811 with the AS delivered core version of threads and 5.8.8/AS817 with the latest cpan versions of threads & threads::shared. And that's relying upon Perl's garbage collection alone.

More information needed. (I still suspect Data::Dump)

#! perl -slw use threads; use threads::shared; print $threads::VERSION; print $threads::shared::VERSION; print $]; @th = map{ threads->new( sub{ while( 1 ){ local %h = ( 1 .. 10000 ); sleep 1 } } ) } 1 .. 100; $_->join for @th __END__ c:\test>junk9 Name "main::h" used only once: possible typo at c:\test\junk9.pl line +13. 1.05 0.92 5.008006 Terminating on signal SIGINT(2) c:\test>\AS817\perl\bin\perl5.8.8.exe junk9.pl Name "main::h" used only once: possible typo at junk9.pl line 13. 1.1201 0.99 5.008008 Terminating on signal SIGINT(2)

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

Replies are listed 'Best First'.
Re^6: Massive Perl Memory Leak
by wagnerc (Sexton) on Jun 13, 2007 at 20:08 UTC
    Well I just restored %datahash usage to the rebuild of the script and it didn't leak. :) Kind of glad that's not it. I also just restored Data::Dump usage to the rebuild and am testing it now. We'll see if it goes down. I don't remember if I was using Data::Dump in the first place when I noticed the leak. Actually, yes. I remember now, I need Data::Dump to serialize the hash to pass it through a message queue to the database thread.