in reply to Re: memory leak
in thread memory leak

I put Devel::Leak in my outer loop and am not quite sure what I'm looking at - most loops it does this:
old (1): 0 old (1): 0 old (1): 0 old (1): 0 old (1): 0 old (1): 0 old (1): 0 old (1): 0 old (1): 0 old (1): 0 old (1): 0 old (1): 0 old (1): 0 old (1): 0 old (1): 0

but every say 5 iterations it does this:
ew 04EF14D4 : new 04EF14E0 : new 04EF1510 : new 04EF1558 : new 04EF15E +8 : new 04EF16B4 : new 04EF17E0 : new 04EF17F8 : new 04EF184C : new 0 +4EF1858 : new 04EF18A0 : new 04EF1CB4 : new 04EF1CD8 : new 04EF1FF0 : + new 04EF214C : new 04EF21C4 : new 04EF21E8 : new 04EEA73C : new 04EE +A748 : new 04EEA760 : new 04EEA8BC : new 04EEAA3C : new 04EEAA9C : ne +w 04EEAAC0 : new 04EEAB50 : new 04EEABBC : new 04EEACA0 : new 04EEACB +8 : new 04EEAD90 : new 04EEAF1C : new 04EEAFE8 : new 04EEB0CC : new 0 +4EEB138 : new 04EEB21C : new 04EEB228 : new 04EEB234 : new 04EEB240 : + new 04EEB2D0 : new 04EEB378 : new 04EEB3D8 : new 04EEB3E4 : new 04EE +B480 : new 04EE38BC : new 04EE394C : new 04EE3970 : new 04EE39B8 : ne +w 04EE3B50 : new 04EE3DE4 : new 04EE3DFC : new 04EE3EC8 : new 04EE3EE +0 : new 04EE3FAC : new 04EE4018 : new 04EE40C0 : new 04EE4120 : new 0 +4EE41E0 : new 04EE4534 : new 04EE45F4 : new 04EE46F0 : new 04EE4744 : + new 04EDCC7C : new 04EDCEA4 : new 04EDCEF8 : new 04EDCF28 : new 04ED +D528 : new 04EDD690 : new 04EDD6CC : new 04EDD708 : new 04EDD930 : ne +w 04EDD948 : new 04ED6190 : new 04ED61C0 : new 04ED6460 : new 04ED648 +4 : new 04ED6688 : new 04ED66A0 : new 04ED66D0 : new 04ED66F4 : new 0 +4ED6868 : new 04ED69D0 : new 04ED6A54 : new 04ED6B14 : new 04ED6B8C : + new 04ECF238 : new 04ECF364 : new 04ECF5A4 : new 04ECF6AC : new 04EC +F700 : new 04ECF994 : new 04ECFB20 : new 04EFF644 : new 04EF850C : ne +w 04EF87AC : new 04EF8D70 : new 04EF8D88 : new 04EF18F4 : new 04EF1A7 +4 : new 04EF1DEC : new 04EF2338 : new 04EEA700 : new 04EEAD48 : new 0 +4EEAE08 : new 04EEAE2C : new 04EEB234 : new 04EEB2E8 : new 04EE3C64 : + new 04EE3D90 : new 04EE42C4 : new 04EE454C : new 04EE46D8 : new 04ED +69AC : new 04F064F8 : new 04EFF008 : new 04EFF344 : new 04EFF854 : ne +w 04EFFE60 : new 04EF83C8 : new 04EF84D0 : new 04EF87AC : new 04EF881 +8 : new 04EF181C : new 04EF1894 : new 04EF18D0 : new 04EF1A20 : new 0 +4EEA994 : new 04EEB15C : new 04EEB180 : new 04EEB4F8 : new 04EE4294 : + new 04EE46FC : new 04EDCEE0 : new 04ED5E60 : new 04F0D36C : new 04F0 +6744 : new 04EFF008 : new 04EFF44C : new 04EFF6EC : new 04EFF794 : ne +w 04EFFB54 : new 04EFFCE0 : new 04EF8524 : new 04EF87E8 : new 04EF1C3 +C : new 04EF1D38 : ... etc .... old (1): 0 old (1): 0 old (1): 0 old (1): 0 old (1): 0 old (1): 0 old (1): 0 old (1): 0 old (1): 0 old (1): 0 old (1): 0 old (1): 0 old (1): 0 old (1): 0 old (1): 0

Any insight would be appreciated!

Replies are listed 'Best First'.
Re^3: memory leak
by BioLion (Curate) on Aug 26, 2009 at 18:58 UTC
      i think i found the issue;

      let's say getObjects opened and closed a db connection every call to get its data, using Win32::ODBC. This appears to be causing the memory to grow. here is an example:
      use Win32::ODBC; my @objects = (); my $i = 0; while($i < 20000000) { @objects = getObjects(); # do something $i += 100; } sub getObjects { my @objects = (); my $object = undef; my $db = undef; my $i = 0; $db = new Win32::ODBC("DSN=dsnname;UID=username;PWD=pw%") || die; for($i=0; $i<100; $i++) { #$object = new Obj("name_$i", $i); $objects[$i] = $object } if($db) { $db->Close(); } return @objects; }
      I tried adding $db = undef; with no luck. does $db not get re-used every call??

        I don't have any experience with Win32::ODBC, but looking at the release notes, $db shouldn't be the problem, as it is only storing the connection, not the data, and it falls out of scope at the end of the scope anyway, so it should be garbage collected and returned to the pool anyway.

        Are you sure that $db is being closed every time? Your script will die if the connection is not made, so there is no need for the if ($db){...} loop.

        I would also get rid of the global variable @objects and put the definition inside the while($i < 20000000) loop. This way you know that the array and it's contents are definitely out of scope with each iteration of the while loop.

        Perl garbage collection is refernce count based, (as i am sure everyone has already said), so if your memory really does continue to grow even though you have tightened the scoping, and are sure your new Obj("name_$i", $i) has not circularities, then my suspiscion would be that the ODBC memory is not being freed to the memory pool allocated to perl, but even this shouldn't be an issue because you are not making new DBs or adding to the DB right?

        One last thought, just to make sure your objects are being destroyed, you could add some debugging code along the lines of :

        use Win32::ODBC; use warnings; use strict; my $i = 0; while($i < 20000000) { print "***NEW ITERATION \$i = $i\n"; my @objects = getObjects(); # do something $i += 100; } sub getObjects { my @objects = (); my $object = undef; my $db = undef; my $i = 0; $db = new Win32::ODBC("DSN=dsnname;UID=username;PWD=pw%") || die " +Failed to establish connection\n"; for($i=0; $i<100; $i++) { #$object = new Obj("name_$i", $i); $objects[$i] = $object; print "Object ".( $object->to_string() )." created.\n"; } $db->Close(); return \@objects; } ################# IN Obj class definition ########### sub DESTROY{ my $self = shift; print "".( $self->to_string() )." biting the dust.\n"; }

        This way you should be able to be really sure that all Obj objects are being destroyed each iteration.

        Hope this helps, but i am not sure! Good luck!

        Just a something something...