in reply to Re: Threads memory consumption is infinite
in thread Threads memory consumption is infinite

This is starting to ring a bell, I've seen this with Tk, you need to reuse your threads, because the refcount is so complicated, that Perl won't free the thread's memory when it's undefind. What I do in Tk-with-worker-threads is create 3 reusable worker threads( you would want to create 20). My example looks complex because I use a bunch of hash names, but you can simplify it for your purposes. What I essentially do, is store the available threads in an array @ready. I shift off a thread as I need a thread to work, and when it is done working, I push it back onto the @ready array for next use. This way, only 20 threads ever get created, and only 20 max can run at a time. It works to conserve memory by reusing the threads. You only need to join the threads once, when exiting the program.

So instead of spawing a new one as an old one dies off, push the dying one onto the @ready array, and shift one off for the next thread. Believe me, it works and is solid, and avoids the refcount problem.( To avoid confusion, when you get a worker, a little popup appears that spawns an xterm, just close the xterm to start the thread running. On win32 change the cmd to something that works. I did this to give a visual indication that the thread was actually doing something).

I'm not really a human, but I play one on earth CandyGram for Mongo
  • Comment on Re^2: Threads memory consumption is infinite

Replies are listed 'Best First'.
Re^3: Threads memory consumption is infinite
by BrowserUk (Patriarch) on Jun 10, 2008 at 20:02 UTC
[reply]
      You came up with the ......

      I plead the 5th Amendment. :-)

      I'm not really a human, but I play one on earth CandyGram for Mongo
[reply]

        awesome, that sounds like its exactly what i want to do and a perfect solution. i'll let you know how it works out

[reply]
Re^3: Threads memory consumption is infinite
by Godsrock37 (Sexton) on Jun 11, 2008 at 14:44 UTC

    quick question... how do i start/stop each thread?

    in other words... so the thread finishes its job and i will immediately have another job for it. i dunno, i like the @ready strategy and reusing the threads, that seems like its exactly what i need, but i dont know how to implement it.

    i looked at your code but, like you said, its a little more than i need. i dont need to share any data between threads except the queue which already works

    can i send you a PM or an email or something? i love perlmonks but its a little bit of a low bandwidth form of communication

[reply]
      If you look closely at my thread's code block, you will see it is wrapped in a while loop.....I call it a sleeping thread, and you control it thru shared variables. When you create the thread, it keeps looping(and sleeping a bit) until it gets a go variable change. Here is a simple example of a sleeping thread. As you can see from my other complex example, you need to keep track of everything in hashes if you have more than one threads. It also helps to have an event-loop system, so you don't need difficult while() loops in main to watch the threads, but a clever while loop will work. SeeRoll your own Event-loop and Readkey with timer using Glib

      Don't forget, you can modify that thread's while loop to suit your purposes, you just need to be creative to suit your exact situation. Also remember, you can share filhandles across threads thru the fileno. So you might want to communicate by print to a filehandle in the thread, and reading it in main. 

      #!/usr/bin/perl
use strict;
use warnings;
use threads;
use threads::shared;

my $data:shared = '';
my $go_control:shared = 0;
my $die_control:shared = 0;

my $thr = threads->new(\&excecute);


print "press a key to start thread\n";
<>;
$go_control = 1;


for( 1.. 10){
   print "\t\t$data\n";
   select(undef,undef,undef,.25);
}


print "press a key to stop thread\n";
<>;
$go_control = 0;

print "press a key to kill thread\n";
<>;
$die_control = 1;
print "press a key to exit\n";
<>;


sub excecute{
      
   my $count;
  
   while(1){
      
      if($die_control){
             print "thread finishing\n"; 
             return} 
      
      #wait for $go_control  
      if($go_control){      
          
                 if($die_control){
                    print "thread finishing\n"; 
                           return} 
               
               $data = $count++;            
               print $data,"\n";
           select(undef,undef,undef,.25);
         
      
        }else{ select(undef,undef,undef,.25); }# sleep until awakened 
   }
    
return;
}

      [download]
      I'm not really a human, but I play one on earth CandyGram for Mongo
[reply]
[d/l]

In Section Seekers of Perl Wisdom