in reply to Re^6: Threads question
in thread Threads question

I have to agree with you that it all works pretty well, that is why I like using threads. I did some more realistic testing and found that the threads were collecting garbage alot better than I thought they would. In this more realistic script, the memory holds steady. I don't know why my previous experience with threads showed otherwise..... maybe it was because I was using guis, (non-thread-safe Tk) or older Perl versions? Possibly I was leaving non-detached threads in an unjoined condition, OR had threads which were hanging somewhere in their code? 
#!/usr/bin/perl
use warnings;
use strict;
use threads;
use threads::shared;

my $count = 0;

while (1) {
        $count++;
        print "thread $count started\n";         
        my $t = threads->new(\&do_my_thing, $count)->detach;
        sleep 5;
    }


sub do_my_thing {
    
    use LWP::Simple;
    use Data::Dumper;
    use HTML::TokeParser::Simple;
   
   my $get = 'http://google.com';
   my $page=get( $get );

   my $p = HTML::TokeParser::Simple->new( \$page );
    while ( my $token = $p->get_token ) {
       # This prints all text in an HTML doc (strips HTML)
           next unless $token->is_text;
       print $token->as_is;
      }

    my $val = shift;
    select(undef,undef,undef,.1);
    print "thread $val ended\n";         
}

[download]
I'm not really a human, but I play one on earth. Cogito ergo sum a bum

Replies are listed 'Best First'.
Re^8: Threads question
by BrowserUk (Patriarch) on Jun 30, 2007 at 17:13 UTC

    You might also like to try this. On my system it settles down to a steady number of threads within 5 or 10 seconds and thereafter creates and destroys threads at the rate of about 7/second with the memory usage fluctuating a few Kb either side of 30MB.

    I left this running whilst I ate (be warned, it thrashes the cpu). The thread cycles had reached over 12000 by the time I returned and the memory usage was still locked at ~30MB. And that was with 5.8.6. If there is a leak there, it's an extremly slow one.

    If your processor is faster or slower than mine, you might need to tweak the sleep parameter within the thread to get it to achieve equilibrium quickly. 

    #! perl -slw
use strict;
use threads;
use Thread::Queue;
use threads::shared;
use Time::HiRes qw[ time ];

$|++;

our $N ||= 1000;
our $M ||= 10;

our $aClonedGlobal          = 12345;
our $aSharedGlobal :shared  = 12345;
my $aClonedLexical          = 12345;
my $aSharedLexical :shared  = 12345;

my $count :shared = 0;
my $running :shared = 0;
my $Q = new Thread::Queue;

sub thread {
    { lock $running; ++$running }
    my $tid = threads->self->tid;
    my( $some, $thread, $local, $vars ) = (12345) x 4;
    require Carp;
    require IO::Socket;
    require LWP::UserAgent;
    $Q->enqueue( threads->tid );
    sleep $M/10;
    { lock $running; --$running };
    return 1;
}

threads->create( \&thread ) for 1 .. $M;

my $start = time;
while( 1 ) {
    threads->object( $Q->dequeue )->join;
    threads->create( \&thread );
    printf "\r%d\t (%7.3f/sec)\t", ++$count, $count / ( time() - $star
+t );
    sleep 0;
}

    [download]
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]

In Section Seekers of Perl Wisdom