in reply to Strawberry perl memory leak

Suitably adjusted to run under 5.8.x, I see the same behaviour (continuous memory growth) under both AS811/5.8.6 and AS1002/5.10. The memory growth is much faster with 5.10. After 100 forks, 5.8.6 is consuming 120MB. At the same point 5.10 is consuming 260MB.

I suspect that the root cause is that the old threads are not being joined until the loop is completed, so that the old thread carcasses (stack and heap allocations) are just accumulating in memory (much like *nix zombies), until their return values are either collected or discarded with join or detach, and that isn't done until the wait_for_all method is called.

Speculation, but it fits the symptoms.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

Replies are listed 'Best First'.
Re^2: Strawberry perl memory leak
by EvanCarroll (Chaplain) on May 15, 2008 at 17:24 UTC
[reply]
      Any solution in all of that?

      Not one you'll like. I reached the conclusion a long time ago that using fork on Win32 is a pointless exercise.

      The emulation simply isn't accurate enough, nor complete enough to enable transparent cross-platform solutions. And if you need to put conditional code in to test and account for Win32 limitations and variations, then it's better to move to a full Win32 native solution.

      See this Re: Question: Fast way to validate 600K websites from a few days ago that does almost exactly what you are trying to do using Parallel::ForkManager. It works (on Win32 and anywhere where threads works), and the code is hardly more complex.

      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

In Section Seekers of Perl Wisdom