in reply to Re^18: Strange memory leak using just threads (forks.pm)
in thread Strange memory leak using just threads

Let me quote:

And the tests are not correct BTW. Time required for creating new thread depends on the size of all existing variables.
Yes. That is an annoying detail of the ithreads implementation. But, it is quite easy to avoid; you just spawn your workers early, and have them require rather than use what they (individually) need.

So, if worker threads require some heavy module, you have to use it before starting these threads (and I don't mean cases where you need just single worker thread), and it may seriously increase worker threads startup time, and it is not easy to avoid.

  • Comment on Re^19: Strange memory leak using just threads (forks.pm)

Replies are listed 'Best First'.
Re^20: Strange memory leak using just threads (forks.pm)
by BrowserUk (Patriarch) on Sep 23, 2010 at 15:34 UTC

    Yes. And if every worker fork needs to modify 1 bit on every 4x page of 1 GB of shared data, it's gonna take forever.

    You show me your use-case, and I'll show you mine.

[reply]
      And if every worker fork needs to modify 1 bit on every 4x page of 1 GB of shared data, it's gonna take forever.

      Rubbish

      use strict;
use warnings;
use 5.010;
use Time::HiRes qw(time);

my $data = 'x' x 500_000_000;
my $t = time;
$data =~ tr/x/y/;
say time - $t;
__END__
0.469923973083496

      [download]
      use strict;
use warnings;
use 5.010;
use Time::HiRes qw(time);

my $data = 'x' x 500_000_000;
my $t    = time;
unless (fork) {
    $data =~ tr/x/y/;
    exit;
}
wait;
say time - $t;
__END__
0.765344142913818

      [download]
      use strict;
use warnings;
use 5.010;
use threads;
use Time::HiRes qw(time);

my $data = 'x' x 500_000_000;
my $t    = time;
threads->create(
    sub {
        $data =~ tr/x/y/;
    }
)->join;
say time - $t;
__END__
1.29563307762146

      [download]
[reply]
[d/l]
[select]

        That's not quite what I said is it :)

        See how you fare with this: 

        use strict;
use warnings;
use 5.010;
use Time::HiRes qw(time);

my $data = 'x' x 500 * 1024**2;

my $t    = time;
for my $n ( 1 .. 100 ) {
    unless (fork) {
        substr( $data, 4096 * $_ + $n, 1 ) |= 1 for 0 .. 124;
        exit;
    }
}
waitall;

say time - $t;

        [download]
[reply]
[d/l]

In Section Seekers of Perl Wisdom