See Parallel::ForkManager and IO::Pipe for some helpful modules.

Perl does re-use memory internally, and may release memory to the OS, although not reliably (but that's OS dependent and not as bad as it sounds in the vast majority of cases).

If you are seeing your program's memory usage increase over time, it's probable you have a memory leak. Setting a hash to undef doesn't guarantee that all references will be garbage-collected, recursively, which is what you might expect. Also, keep in mind undef is not free(); it doesn't de-allocate anything.

First, as is common with GC systems, Perl maintains a reference count to each reference. If you have, say, a map of keys to array refs, and you delete the hash, the arrays will still exist if (and only if) some other thing still references them.

The second bit, which may sound more arcane, is that if, anywhere in your data, you have circular references ($a == $b, but possibly assigned through a longer assignment path as in a circular linked list for example), the memory for $a and $b will never be reused, even if nothing else references $a or $b, unless you break the circular reference.

Usually, breaking circular references isn't obvious and is potentially very expensive. See Scalar::Util's weaken($ref) function, however, for one approach.

If neither of those things seem to describe your problem, perhaps you could post some short sample code for us to have a look at.

I'm asking this becuase undefing the hash or reseting ( undef %hash or hash = () ) doesn't release the memory.

Prove it! (Ie. demonstrate your claim by posting working code; because what you claim is not normally true.)

Should be obvious enough... Emptying or undef'ing a hash might or might not set the reference-count of the objects in question to zero. And even if it does, garbage collection might not occur immediately. So the statement is quite pragmatically correct: you can zap a hash, check the memory allocation immediately thereafter, and find that nothing at all has changed. Yet. If the OP finds that memory is not being released, odds are quite good that there are circular references going on here which haven't been weakened.

Should be obvious enough...

Guess on if you like. I'll wait for the evidence.

---

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

RIP Neil Armstrong

Welcome nowise,

I had a similar problem about a year ago. I had a hash that just grew and grew and it turned out to be a scoping problem. I created the hash in the main script and was trying to clear it in a subroutine. What I cleared was an empty hash and returned to the main script without touching the 'real' hash.

If this isn't what your doing, then you'll have to show some code. The '%Hash' is very powerful and very reliable, so it's probably something your doing wrong. But that's how we learn.

Good Luck!

In order to describe it more clearly , i have 1 example below, in which i have created 1 big hash-of-hash. Although i have put only 2 HOH inside %TV overhere ( just for clarity sake ), but in my code i have it 300 or more HOH ( while incrementing the key e.g jetsons1, jetsons2... ) in order to check the memory usage.

 #!/usr/bin/perl5.8
use Proc::ProcessTable;
#use Tie::File::AsHash;
[download]

 my %hash= ();

%TV = (  flintstones =>{ series   => "flintstones",
                          nights   => [ qw(monday thursday friday) ],
                        members    => [ { name => "fred",    role => "
+lead", age  => 36, },
                                        { name => "wilma",   role => "
+wife", age  => 31, },
                                        { name => "pebbles", role => "
+kid",  age  =>  4, },
                                      ],
                       },
         jetsons1    => { series   => "jetsons",
                         nights   => [ qw(wednesday saturday) ],
                         members  => [ { name => "george",  role => "l
+ead", age  => 41, },
                                       { name => "jane",    role => "w
+ife", age  => 39, },
                                       { name => "elroy",   role => "k
+id",  age  =>  9, },
                                     ]
                       },
[download]

 sub memory_use {
    my $t = new Proc::ProcessTable;
    foreach my $got ( @{$t->table} ) {
        next if not $got->pid eq $$;
        return $got->size;
    }
}
[download]

  my $count = 10;
  my $key = 'test';

  #tie  %hash, 'Tie::File::AsHash', 'abc.csv', split => ':' or die "Pr
+oblem tying %hash: $!";

  foreach my $family (keys %TV) {
      $count++;
      my $record = "";
      my $rec = $TV{$family};
      my $key = $family . $rec->{series};

      for $person ( @{ $rec->{members} } ) {
          $record .= $person->{name}."," . $person->{role}. ","
                                         . $person->{age} . ","
                                         . $rec->{series}   . ","
                                         . $rec->{nights}[0];
      }

      print "\n Key:    $key \n";
      print "\n Record: $record \n";


      $hash{$key} = $record;

      if ( $count%10 == 0 ) {
          sleep(10 );
          print "\n Record counter => [$count] \n";
          print "\n Process Status:" . ( ( memory_use()/1024 )/1024 ) 
+."MB \n";
          %hash = (); # if we comment it memory constant
      }
  } #----for 1----#
[download]

nowise,

I ran your code (after a little clean-up) out to:

  Record counter => 335700
[download]

 '/bin/ps -o rssize -p $$'
[download]

Some suggestions:

• perl5.8.8 is rather old.
• use strict and use warnings ( You need more 'my's in your code
• Consider using a 'log' file for incrementing information.

From this test, your problem isn't the hash growing. It could be your system or the system's perl that's giving you the problem.

Good Luck!

Update: I remembered I had some AIX systems with perl5.8.8 and I ran the code to 'Record counter => 10320' and the memory usage stayed as '1940MB'. So it doesn't seem to be a problem on the AIX perl5.8.8 either

Update2: My "MB" in above should be "KB" as the way 'top' and 'ps' return the information.

i donot know how the system or perl is giving the error.

can you please provide some more details on it.

i had read somewhere that perl should be compiled with the "usemymalloc=y" compiler flag to release the memory.

when i ran the perl -V on the commandline i could see this flag set to "y".

Also the only contributing factor to increase in the process memory , seems to be because of hash.

please see below the memory increase in the process ,it remain constant to some point , and then gaian start increasing :

Record counter => 20 Process Status:6.328125MB Record counter => 30 Process Status:6.4296875MB Record counter => 40 Process Status:6.4375MB Record counter => 50 Process Status:6.4375MB Record counter => 60 Process Status:6.4453125MB Record counter => 70 Process Status:6.4453125MB Record counter => 80 Process Status:6.4453125MB Record counter => 90 Process Status:6.453125MB Record counter => 100 Process Status:6.453125MB Record counter => 110 Process Status:6.453125MB Record counter => 120 Process Status:6.453125MB Record counter => 130 Process Status:6.4609375MB