Well, it seems you found another reason to be suspicious of threads, especially if you plan to use the script on multiple unknown platforms.

Hopefully faxm0dem has reported the bug to p5p, and as the testcase is so simple, it may get included in the testsuite. It so, it should prevent changes that cause memory leaks from getting through into new releases. Which if the problem has been fixed in 5.8.9 and the 5.10 branches, could make for a more reliable future for iThreads.

And maybe that is good reason to drop some of the old superstitions.

good reason to drop some of the old superstitions.

Yeah, but the problem is that there will always be those old versions of Perl installed somewhere, and even if 5.9 fixes it, all those obsolete 5.8 installs will linger around for years...... keeping the superstitions alive. So if you plan to distribute the scripts to unknown platforms and Perl versions, it may behoove programmers to play it safe, and try to reuse threads, which does work on 5.8. (At least for a few years).

---

I'm not really a human, but I play one on earth Remember How Lucky You Are

How would you suggest to recycle threads in practice?

In simple terms, instead of creating a new thread to process each piece of asynchronous work, you start by creating a pool of threads that sit and do nothing until your main thread passes them a piece of work to do. They then process that piece of work and instead of dieing when it is complete, they go back to waiting for another new piece of work.

Here's a simple example:

#! perl -slw
use strict;
use threads;
use Thread::Queue;

sub worker {
    my $Q = shift;
    while( my $workItem = $Q->dequeue ) {
        printf "[%d] Processing workiterm '%s'\n", threads->tid, $work
+Item;
        sleep rand( 2 );  ## process $workitem
    }
}

our $WORKERS ||= 10;

my $Q = new Thread::Queue;
my @threads = map{ threads->create( \&worker, $Q ) } 1 .. $WORKERS;

while( <> ) { ## Get workitems from stdin
    chomp;
    $Q->enqueue( $_ ); ## And queue them to the worker pool
}

$Q->enqueue( (undef) x $WORKERS ); ## Signal no more work
$_->join for @threads; ## Wait for tehm to finish;

exit; ## Done
[download]

You'd use it like this:

> ls * | perl -s tdemo.pl -WORKERS=5
[1] Processing workiterm '2of12inf.dic'
[1] Processing workiterm '2of12inf.txt'
[1] Processing workiterm '3'
[4] Processing workiterm '345241'
...
[download]

Or like this:

>tdemo -WORKERS=3 work.dat
[2] Processing workiterm '00001'
[1] Processing workiterm '00002'
[2] Processing workiterm '00003'
[1] Processing workiterm '00004'
[2] Processing workiterm '00005'
[1] Processing workiterm '00006'
[3] Processing workiterm '00007'
[2] Processing workiterm '00008'
...
[download]

Of course, this doesn't do very much--just prints and sleeps--but this simple basic structure can be used to service a huge variety of different concurrent programming tasks. Not all, in particular, socket servers require a somewhat modified approach, but still a good proportion of concurrency tasks can be handled this way. And as you only create a limited number of threads, you gain performance because you're not constantly discarding old threads only to replace them near identical new ones.

And, the bit that is very relavent in the context of this thread, if there are any small, per-thread memory leaks, they never become an issue, because you are creating so few threads.

To be able to tailor a reply to your particular situation, you'd have to tell us what it is you are currently doing with those 10,000 threads you are creating :)

---

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

Thank you for your very helpful example.

To be able to tailor a reply to your particular situation, you'd have to tell us what it is you are currently doing with those 10,000 threads you are creating :)

Well, in my real world code, I'm only creating new threads (~10) every 15 seconds. Each thread is collecting system data, which is later sent over to a collector. So I'll most certainly work as you suggested, namely recycle the threads.

See Reusable threads demo for a bare bones example, and notice TGI's improvements.

---

I'm not really a human, but I play one on earth Remember How Lucky You Are