also, If i am not mistaken, once $mbox gets undef, the memory was allocated by it goes back to the memory pool.

That is a common mistake, and I usually find a way around it, although I don't fully understand it myself.

The memory is returned when there is no further references to the hash, anywhere in your code. There are also other complications, where Perl will release the memory for reuse, but it won't release it to the system until the program ends. So if you run a small test script filling the hash up to 30 megs, then undef the hash, the Perl script will stay at 30 Megs, but will not increase any further if you reload a second 30 meg hash, it will reuse it's internal allocations.

But it sounds more like you have "hidden internal references" to the hash, probably buried down in one of the modules. It sounds like a good plan, to load a small hash, and save it to a database, the reuse the hash for the next batch; but you have to be very careful about deleting the hash. Do a Super Search for "auto-vivication", you may be getting bit by that. It's possible that somehow, one of the modules is keeping an empty hash element around. Even though it is empty, it is a reference to the old hash, and prevents the memory release.

You have to carefully plan for memory reuse, it won't happen magically by undef'ing things, like it would in C or C++.


I'm not really a human, but I play one on earth. flash japh

In reply to Re: hash and memory usage by zentara
in thread hash and memory usage by Qiang

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.