Re^2: threads: spawn early to avoid the crush.

Yeah. I wish I understood, or one of the guys that know would tell us, where the memory growth actually arises.

If you run this (having substituted a suitable mem routine for your platform), and then play with the various values, it's really difficult to devine where the growth occurs and what controls how much?

#! perl -slw
use strict;
use Data::Dumper;
use threads;
use threads::shared;
no warnings 'misc';

our $N ||= 100;
our $D ||= 1.e5;
our $SHARED;

sub mem {
    my @filler          = 1 .. $D unless @_;
    my @filler : shared = 1 .. $D if @_;
    my( $usage ) =  `tasklist /NH /FI \"pid eq $$\" ` =~ m[ (\S+) \s+ 
+K \s* $ ]x;
    $usage =~ tr[,][]d;
    return 1024 * $usage;
}

my @data        = 1 .. $D unless $SHARED;
my @data:shared = 1 .. $D if $SHARED;

printf "start : %6d\n", my $start = mem;
for ( 1 .. $N ) {
    my $thread = threads->create( \&mem );
    printf "%3d   : %6d\n", $_, $thread->join;
}
printf "end   : %6d\n", my $end = mem;
printf "Growth:  %6d\n", $end - $start;
[download]

Here are some typical results on my system:

c:\test\fork>..\534254 -N=10 -D=1
start : 3141632
  1   : 3993600
  2   : 4018176
  3   : 4018176
  4   : 4022272
  5   : 4038656
  6   : 4038656
  7   : 4022272
  8   : 4018176
  9   : 4055040
 10   : 4055040
end   : 4034560
Growth:  892928

c:\test\fork>..\534254 -N=10 -D=1 -SHARED
start : 3145728
  1   : 3997696
  2   : 4022272
  3   : 4022272
  4   : 4030464
  5   : 4026368
  6   : 4022272
  7   : 4059136
  8   : 4059136
  9   : 4059136
 10   : 4059136
end   : 4042752
Growth:  897024

c:\test\fork>..\534254 -N=10 -D=1e3
start : 3211264
  1   : 4128768
  2   : 4149248
  3   : 4149248
  4   : 4157440
  5   : 4149248
  6   : 4153344
  7   : 4157440
  8   : 4161536
  9   : 4157440
 10   : 4198400
end   : 4132864
Growth:  921600

c:\test\fork>..\534254 -N=10 -D=1e3 -SHARED
start : 3555328
  1   : 4345856
  2   : 4378624
  3   : 4382720
  4   : 4386816
  5   : 4395008
  6   : 4390912
  7   : 4390912
  8   : 4390912
  9   : 4386816
 10   : 4395008
end   : 4374528
Growth:  819200

c:\test\fork>..\534254 -N=10 -D=1e5
start : 9723904
  1   : 17874944
  2   : 17907712
  3   : 17920000
  4   : 17940480
  5   : 17924096
  6   : 17940480
  7   : 17936384
  8   : 17944576
  9   : 17936384
 10   : 17952768
end   : 11743232
Growth:  2019328

c:\test\fork>..\534254 -N=10 -D=1e5 -SHARED
start : 43819008
  1   : 48697344
  2   : 48721920
  3   : 48726016
  4   : 48726016
  5   : 48730112
  6   : 48730112
  7   : 48730112
  8   : 48734208
  9   : 48738304
 10   : 48738304
end   : 44195840
Growth:  376832
[download]

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

Comment on Re^2: threads: spawn early to avoid the crush. Select or Download Code

Replies are listed 'Best First'.
Re^3: threads: spawn early to avoid the crush. by zentara (Cardinal) on Mar 03, 2006 at 17:48 UTC
It's probably something to do with the ref count problem which has been discussed so much before. Just like in Tk, we know it's a ref count problem, but there is no way to keep track of all the possible refs an object creates. So it all boils down to making an arbitrary rule requiring reusing the object, instead of making new ones. I'm not really a human, but I play one on earth. flash japh	[reply]