in reply to threads: spawn early to avoid the crush.

After reading all the replies, it dawned on me that no one really mentioned the problem of the memory gains that can occur if you use "disposable threads in perl". I've tried a few times to set up threads that I try to undef after joining, or after they have otherwise finished. I found that you need to reuse the thread itself, else memory use will start creeping up. That definitely is something to consider in your "thread-launcher idea". I don't know if it can be done. Threads are like Tk objects, they need to be reused, and don't work well in the create-destroy cycle.

I'm not really a human, but I play one on earth. flash japh
  • Comment on Re: threads: spawn early to avoid the crush.

Replies are listed 'Best First'.
Re^2: threads: spawn early to avoid the crush.
by BrowserUk (Patriarch) on Mar 03, 2006 at 17:04 UTC

    Yeah. I wish I understood, or one of the guys that know would tell us, where the memory growth actually arises.

    If you run this (having substituted a suitable mem routine for your platform), and then play with the various values, it's really difficult to devine where the growth occurs and what controls how much?

    #! perl -slw use strict; use Data::Dumper; use threads; use threads::shared; no warnings 'misc'; our $N ||= 100; our $D ||= 1.e5; our $SHARED; sub mem { my @filler = 1 .. $D unless @_; my @filler : shared = 1 .. $D if @_; my( $usage ) = `tasklist /NH /FI \"pid eq $$\" ` =~ m[ (\S+) \s+ +K \s* $ ]x; $usage =~ tr[,][]d; return 1024 * $usage; } my @data = 1 .. $D unless $SHARED; my @data:shared = 1 .. $D if $SHARED; printf "start : %6d\n", my $start = mem; for ( 1 .. $N ) { my $thread = threads->create( \&mem ); printf "%3d : %6d\n", $_, $thread->join; } printf "end : %6d\n", my $end = mem; printf "Growth: %6d\n", $end - $start;

    Here are some typical results on my system:


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      It's probably something to do with the ref count problem which has been discussed so much before. Just like in Tk, we know it's a ref count problem, but there is no way to keep track of all the possible refs an object creates. So it all boils down to making an arbitrary rule requiring reusing the object, instead of making new ones.

      I'm not really a human, but I play one on earth. flash japh