1. Are you continuously adding elements to a hash or array? For example, if you keep doing something like $foo[$i++] = $bar;, you keep telling Perl to increase the size of @foo.
2. Do you have any circular references? They are a cause of true memory leaks.
3. Are you scoping things as much as possible? Do you have any large global hashes/arrays that you could scope a little smaller?
4. Are you constantly loading new classes/modules? I was on a project for a OO application that would dynamically load the classes it needed as it needed to. Every time that happened, the memory usage would increase, but only a fixed amount.
5. Are you fooling around with deleting from %INC and @INC? I'm not sure what that would do, but it probably won't actually remove the initial require from RAM.

------
/me wants to be the brightest bulb in the chandelier!

Vote paco for President!

Heh, well, we do all those things, but they should be controlled.
1. Yes, in various places, but they should all be limited in one way or another (either they go out of scope at various points or they only have a limited number of elements in them).
2. We do have one module that uses circular references, but using Weakref, that shouldn't be a problem.
3. Yes, scoping on the project is very tight.
4. This is the thing that I'm worried about. We're constantly loading new classes, dymanically created from database information. Now, theoretically, we have a finite number of classes, so the memory usage from loading these things should be finite itself (and that limit should be hit rather quickly). However, I'm not positive this is the case. Anyway know much about the finer points of this?
5. No. Thanks for the help.

To reply to a few points:
1. Be very careful. Simply because something is out of scope does not mean that it's been garbage-collected. Directly from the Camel (3rd ed., p.223):

   Lexical variables don't get automatically garbage collected just because their scope has exited; they wait to get recycled until they're no longer used, which is muych more important. To create private variables ...

   I would suspect that this is a minor problem, but probably not the whole issue.

2. As for the constantly loaded classes ... I'm going to guess that you're creating the classes using some sort of AUTOLOAD(), eval, or combination of the two. If that's the case, these classes are probably taking quite a chunk of memory. Now, by doing it this way, you're taking away any possible compile-time optimizations that the compiler could do. In addition, I think that once you use AUTOLOAD() once, the compiler will allocate additional memory every time, just as if you used $` or $' in regexps.

   I think that this is the culprit. I would suspect that it's not that you're loading 10^12 classes, but that each class is taking a bigger chunk than you realize.

------
/me wants to be the brightest bulb in the chandelier!

Vote paco for President!

We're constantly loading new classes, dymanically created from database information. Now, theoretically, we have a finite number of classes, so the memory usage from loading these things should be finite itself (and that limit should be hit rather quickly). However, I'm not positive this is the case. Anyway know much about the finer points of this?

Are you cleaning up the symbol tables after you're done using these new classes, or are they being re-used? We found in Class::Prototyped that each new class takes up around 1.5-2K of memory; more (of course) with methods. You can clean up the symbol table when you're done with code like this (assumes $package does not contain '::'):

no strict 'refs';
foreach my $key ( keys %{"$package\::"} ) {
    delete ${"$package\::"}{$key};
}

# this only works because we're not a multi-level package:
delete( $main::{"$package\::"} );
[download]

Data::Dumper is a good tool to help track down circular references. If you see something like:

$VAR1 = \$VAR1;
or
$VAR1 = [ $VAR1 ];
or
$VAR1 = { key => $VAR1 };
[download]

Then you know you have something to fix.

Also, I've experienced a few problems w/ memory leaks in perl 5.6.0 and linux. e.g.:

eval "use Any::Module";
[download]

Leaks memory for me, and doesn't appear to with 5.6.1. But I haven't done enough testing to be definitive about that. Someone else might be able to tell you more about this.

Are you doing anything with subroutine references? It's possible to chew up RAM by using things like Error.pm's try/catch syntax if you aren't careful to avoid accidental closures.

You might want to check out the mod_perl guide for additional information on Perl memory management.

perrin++

Just to be clear, the memory used directly by the lexical is not freed but if, for example, the lexical contains the last reference to something, then that something will be destroyed and its memory freed when the lexical's scope is left (or when the last reference to the lexical is destroyed). So this is mostly a problem when dealing with long strings rather than objects.

And the bug with closures never being freed has been found but I don't think any Perls have been released that contain this fix.

        - tye (but my friends call me "Tye")

Just to be clear, the memory used directly by the lexical is not freed but if, for example, the lexical contains the last reference to something, then that something will be destroyed and its memory freed when the lexical's scope is left (or when the last reference to the lexical is destroyed). So this is mostly a problem when dealing with long strings rather than objects.

No, I don't think that's correct. The memory will not be freed, if by freed you mean "available for use by other variables in my program". It will be re-used if you use that lexical again. See this post from Doug MacEachern for more info on this.

I'm not sure the closures thing I was talking about is really a bug; it's just a side-effect of nested subs and closures. If you do this, it will leak like crazy:

my $foo;
sub bar {
  sub baz {
    $foo++;
  }
}
[download]

It's creating a new private copy of $foo for baz() every time. Using Error.pm can sometimes lead people to do this without realizing it, when they use nested try/catch blocks.

See the section on "Debugging Perl memory usage" in perlman:perldebug.

Hope this helps!