Hi,

I am gathering data to a hash and encounter the perl memory freeup issue (memory never return to the system unless I reboot it). I thought that I would use module storable and break down the data collection to steps, then store each collected data at step into local disk to minimize the issue. but that doesn't seems to get any better. 

use File::Find;
use Storable qw(store retrieve);

my $mbox = {}

foreach my $usr (@usr_dirs) {
    chomp $usr;
    my $usr_dir = $base_dir."/".$usr;

    undef $mbox;
    $mbox = retrieve($storable_file) if -e $storable_file;

    find(\&wanted,$usr_dir);
    store $mbox, $storable_file or die "Can't store !\n";
}

[download]

the find subroutine does all the data collection. before coming up with the above code, I stored all data into $mbox hash once then saved it to the local disk, the memory usage is really high. I was hoping by breaking it down by storing / retriving / storing , the memory usage issue could be resolved.

also, If i am not mistaken, once $mbox gets undef, the memory was allocated by it goes back to the memory pool. so shouldn't the overall maximum memory usage = max(batch_1_mem_usage,batch_2_mem_usage, batch_n_mem_usage) ?

In reply to hash and memory usage by Qiang

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.