package Leaker;
use Data::Dumper;
$Data::Dumper::Indent = 1;
%CLASS;

*CORE::GLOBAL::bless = sub {
  my $obj = shift;
  my $class = shift || caller();
  if (exists $CLASS{$obj}) {
    delete $CLASS{$obj};
  }
  $obj = CORE::bless($obj, $class);
  if ($class =~ /ObjClass/ or (caller())[1] =~ /file|names|here/) {
#    print "$obj created\n";
    $CLASS{$obj} = join "\n\t", map {join ":", caller($_)} 1;
  }
  $obj;
};

sub UNIVERSAL::DESTROY {
  my $obj = shift;
  if (exists $CLASS{$obj}) {
#    print "$obj destroyed\n";
    delete $CLASS{$obj};
  }
}

sub Leaked {
  print Dumper(\%CLASS);
}

1;
[download]

If all else fails you can just restart yourself with:

exec($0, @ARGV);
[download]

I would like to try your code out but am a little confused on how to use it? below is an oversimplification of code I am using for debuging? how would I implement your example in this case. Thanks Robert

#!/bin/perl
use strict 'vars';
use Utils;

while(1){
        my $utlp = new Utils();
        $utlp->connect();
        $utlp->Log("database connected.\n");
        $utlp->disconnect();
        $utlp->Log("database disconnected.\n");
        sleep(5);
}
[download]

I would suggest starting with Devel::Leak. What I offered is mostly for use once you know that you are leaking Perl objects, in a class without a DESTROY, and you need to figure out where. In which case I would use it like this:

#!/bin/perl
use strict 'vars';
use Leaker;
use Utils;

%Leaker::CLASS = ();
while(1){
        Leaker::Leaked();
        %Leaker::CLASS = ();
        my $utlp = new Utils();
        $utlp->connect();
        $utlp->Log("database connected.\n");
        $utlp->disconnect();
        $utlp->Log("database disconnected.\n");
        sleep(5);
}
[download]

And then I would run setting different filters in the logging done in the overidden bless in Leaker to narrow things down.

It won't work if the leaked things aren't objects, if they are created at an XS level, or if they have their own DESTROY method. (It would be possible to hack around the latter issue with some Perl wizardry, and I pointed you at example source-code you could find the necessary technique in.)

For me it was useful because I was leaking a big complicated object which had lots of other objects in it. I didn't care to know details about all of the components, I just wanted to figure out why the base object was not going away. It may or may not be useful for you.

I would recommend trying Devel::Leak first.

One thing you can try is an explicit undef on any lexicals holding large amounts of data. This is supposed to release the memory back to the general pool for Perl (although not back to the OS on most systems). Also, make sure you are using references whenever passing around large chunks of data to subroutines.

You may be having problems because of your DBD driver. I've seen some versions of DBD::Oracle exhibit bad behavior in terms of memory, mostly due to the constantly changing OCI libraries they depend on. You might want to try stripping out the DBI stuff (or replacing it with dummy calls) and seeing if that makes your memory problem go away. There are configuration parameters you can tweak if this is the problem.

the only thing we are doing to the databases from this script is connecting and then disconnecting. Yes I am using strict 'vars'? Yes code is running in it own block and I declare the vars using my? I believe strict for the most part checks for this - please correct me if I am wrong? Thanks much Robert

As a way of testing out your problem, I would duplicate the script and comment out some of the database code (and run it against a copy of production), just to see where this memory loss is occurring.

--t. alex

"Of course, you realize that this means war." -- Bugs Bunny.

Just strip down your script to do one pass at the monitoring, then let it exit. Set up a crontab entry which will run your script at specified intervals. For example, to run your monitoring script every 2 minutes 7x24:

% crontab -e
[download]

*/2 * * * * /path/to/monitor_script.pl >>/tmp/mon_logfile 2>&1
[download]

Notes:

• The code (mine) is not optimized and can be improved.
• Again, as tilly states, you should document the use of the cron job.

--Jim

In years past Oracle client libraries were notoriously leaky. In Oracle 7 it was possible to leak large chunks of memory simply by repeatedly connecting and disconnecting from the database. I've heard third-hand reports of leaks with Oracle 8 libraries, but have no first-hand knowledge.

Try this: To rule out Oracle as the culprit, set up a script that does nothing but connect and disconnect, and note its memory profile over a few hours of running.

Next, set up a script that repeatedly issues the same queries in the same that your production script does, but without storing any results in Perl data structures (this should rule out your code as the source of the leaks). If this doesn't demonstrate a leak, the culprit is your code. If it does show a leak, try some of the advice above (finish your queries, etc.)

Oh, and verifying that you're running the latest Oracle client libraries is usually a good idea.

Are you pushing scalars on an array? Do you also re-use the array without letting it go out of scope inbetween?

Unless you also pop that array, you should 'empty' it.

$#my_array=-1;
[download]