in reply to Does Perl have garbage collection mechanism and how it performs?

Unfortunately, the current (as of 5.8) state of the ithreads implementation in perl still has some bugs, and the garbage collection of scalars shared implicitely through a shared array seems to be one of them. I don't know the ins and outs of it, nor if or when it is likely to be cured. From what I've read elsewhere, the bug seems to be a "perl thing", rather than relating to any particular platforms.

You can, at least in my very crude imperical tests, slow down the rate of growth by using Thread::Queue as the mechanism for distributing your data to your threads, but it doesn't fix it completely. 

use 5.008;
use strict;
use threads qw(yield);
use threads::shared ();
use Thread::Queue;
use Time::HiRes qw(sleep);
$|++;

my $Q = new Thread::Queue;
$Q->enqueue( 1 .. 1000 );

my $sub1 = sub {
    my $a;
    while (1) {
        $Q->dequeue;
        yield;
    }
};

my $t1 = new threads $sub1;
my $t2 = new threads $sub1;
my $t3 = new threads $sub1;

$t1->detach;
$t2->detach;
$t3->detach;

while(1) {
    # readFile();
    # ...

    $Q->enqueue(1..1000) unless $Q->pending;
    print $Q->pending;
    yield;
}

[download]

The other possibility, and it's not a good one, is to not detach your threads, but rather to occasionally join them and spawn a new one to replace it. The leaked memory seems to be released back to the OS once the thread is joined. I can't offer you a good designed for this as I haven't yet worked one out for myself. This is a very unsatisfactory situation, and I wish it were not so.

I also wish that I had the ability to assist in fixing the problem but even if I understood enough to construct a patch, I see no way for me to participate in the process.

Examine what is said, not who speaks.
"Efficiency is intelligent laziness." -David Dunham
"When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller


  • Comment on Re: Does Perl have garbage collection mechanism and how it performs?
  • Download Code

Replies are listed 'Best First'.
Re: Re: Does Perl have garbage collection mechanism and how it performs?
by Joost (Canon) on Jun 18, 2003 at 16:59 UTC
    This is patched in bleadperl, read more here and in here.

    As an aside, you could try using forks instead of threads but maybe you wouldn't on windows :-) 

    -- 
#!/usr/bin/perl -np
BEGIN{@ARGV=$0}s(^([^=].*)|=)()s;
=Just another perl hacker\

    [download]
[reply]
[d/l]

      Thanks for the update Joost. I had seen the start of that discussion before, but not to the point where a patch had been forthcoming.

      Unfortunately, the patch to shared.xs does not appear to have made it onto CPAN yet. Will it ever? Or will it be held back until 5.9 is released?

      That makes it very difficult for those using pre-compiled builds of perl to utilise the patch. Whilst there are several repositories that will undertake building a CPAN module to a binary and making it available via PPM, it's lees likely that the same people will undertake applying patches to core modules and building them for public accessibility with all the inherent risks that entails.

      (Are you (even vaguely) interested PodMaster? :) (Pretty please:)(If I supply the tarball with the patch applied?) (Is there a mechanism for determining all the latest patches that would need to be applied to a given version of a cpan module in order to bring it upto bleedperl status?)

      I've seen the forks module, but it strikes me that as forks are emulated using threads under Win32, using forks to emulated threads using emulated forks is a little....erm....artificial:)

      Examine what is said, not who speaks.
      "Efficiency is intelligent laziness." -David Dunham
      "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller


[reply]
        Looks like you lasped into lisp (say that four times fast) in the middle of that post. :)
[reply]

      This isn't directly aimed at you Joost, but it seemed the most appropriate place to mention that unfortunately, the patch for the shared memory leak doesn't appear to work, leastwise not when applied to either AS 5.8 or to 5.8.0-RC3 freshly downloaded and built for Win32 using BCC.

      There is also new discussion in the thread you linked at RT.perl.org to the same effect.

      My best guess is that bleadperl (whatever and whereever that is?) also contains other patches that aren't a part of the latest version available via ftp/CPAN that are also required. Unfortunately, I don't understand the ticketing system. I tried to access the patch via the #19200 number listed on it and got a permission error and ended up applying the patch through cut&paste from the thread.

      (As an aside, has any other Opera user with a perl.org/rt.perl.org login succeeded in logging in there using Opera? I can get in using IE5.5, but don't like using that other than for compatibility checks.)

      Examine what is said, not who speaks.
      "Efficiency is intelligent laziness." -David Dunham
      "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller


[reply]

In Section Seekers of Perl Wisdom