in reply to Re: Strange memory leak using just threads
in thread Strange memory leak using just threads

I can run this for as long as I like: 

#! /usr/bin/perl
use strict;
use warnings;
use threads;

print "threads: ", $threads::VERSION;
print "perl: ", $];
my @child;
my @list =  1 .. 100;

while (1) {
    my @child = map threads->create("test","$_"), @list;
    $_->join for @child;
}

sub test {
    my ($item) = @_;
}

[download]

And the memory usage is rock steady. And that has been true for a long time.

So, what does that tell you?

Your knowledge is out of date. Your platform--be it the pthreads or memory management or whatever--has an underlying problem that cannot be directly addressed by, nor attributed to, Perl. So stop doing that.

BTW: Your "simple reusable threads demo", is way overcomplicated. And always was.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
RIP an inspiration; A true Folk's Guy

Replies are listed 'Best First'.
Re^3: Strange memory leak using just threads
by zentara (Cardinal) on Sep 20, 2010 at 15:01 UTC
    And the memory usage is rock steady. And that has been true for a long time. So, what does that tell you? Your knowledge is out of date. Your platform--be it the pthreads or memory management or whatever--has an underlying problem that cannot be directly addressed by, nor attributed to, Perl. So stop doing that.

    So what does that tell me? First, his script gained memory and your's didn't. I observe the same behavior. So a script written 1 way in Perl may leak, and written another may not. Furthermore, these 2 sample thread scripts are the ultimate in simplicity. Leakage problems become more pronounced when you start adding objects to threaded programs.

    I have the latest Slackware, and before I believe my slackware c libs are faulty, I have to assume Perl's behavior is somehow causing it.

    Since this memory leak problem keeps coming up, all I can do is to mention that thread reuse does prevent the problem.

    But you are more of a thread expert than me, so if you insist that perl threads do not have memory problems, and it is the underlying c library's fault, I call that a cop out. If threads have no problems with ref counts and memleaks, why did you post threads: spawn early to avoid the crush.? Or is that information from you already outdated?

    I'm not really a human, but I play one on earth.
    Old Perl Programmer Haiku ................... flash japh
[reply]
      First, his script gained memory and your's didn't. I observe the same behavior. So a script written 1 way in Perl may leak, and written another may not

      His original doesn't leak here either. All my changes do is run more threads concurrently in an attempt to amplify any problem should it exist. On my system, it doesn't.

      But Vista and Slackware are fundamentally different systems. Hence my attribution to the platform. If the problem was inherent in the (common) Perl sources, it would manifest itself everywhere. It doesn't. So then you have to start looking at the differences.

      See also, (indirectly via 860733), Dave_the_m's comment.

      I'm pretty sure he runs some version of *nix or other. So, maybe it is not the platform, but how Perl is compiled for a given platform.

      Just attributing it to threads isn't very useful.

      Or is that information from you already outdated?

      4 1/2 years, 2 major releases; 7 or 8 subversion releases. Yes. I would think it is fair to say: that is out of date.

      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
        Please don't tell me I need to start using Microsoft Vista in order to get Perl threads working right. :-)
        I'm not really a human, but I play one on earth.
        Old Perl Programmer Haiku ................... flash japh
[reply]
Re^3: Strange memory leak using just threads
by zwon (Abbot) on Sep 20, 2010 at 15:19 UTC
    Your platform--be it the pthreads or memory management or whatever--has an underlying problem that cannot be directly addressed by, nor attributed to, Perl.

    Strangely this problem doesn't affect C and Python threaded programs.

[reply]

      Then that perhaps leave "or memory management or whatever".

      Of course, a memory management problem might be attributed to Perl if you are compling your perl with use_perl_malloc.

      See 860733, and (indirectly) dave_the_m comment.

      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]
        Now that's interesting. I didn't use use_perl_malloc (as you can see in the later comment from 860733 ), but I did use a bunch of -D's that I really have no idea what they do, they were just what was in the config for the stock 5.10 that came with deb lenny.

        I'm recompiling 5.12.2 now with just -des -Dprefix=$HOME/perl512 -Dusethreads.

        And bonk. It still seems to be leaking memory.

[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom