While it's possible I've missed something, and I'll go check, Devel::Size is generally an accurate representation of the amount of actual memory that perl uses for its internal data structures. This does not mean that it will give you an answer anywhere near what top will give you--far from it in some cases.

It can't, for example, show you the memory that might've been allocated by XS code and not managed by perl. (The expat XML library does this, as do the various DBD modules, for example) At some point I may put in support for a few known modules or libraries (like expat, which some of the XML modules use) but that's a dodgy thing, as it means Devel::Size needs grubby details of other people's internals as well as perl's.

It also doesn't show you memory your process (but not perl) may have allocated but that isn't currently tied up in a perl data structure. Memory allocations are generally done with a particular granularity, often a power of two or a factor of four or eight (depends on the type of request, OS, and processor architecture), and if your perl has requested a chunk of memory smaller than that amount it'll get silently rounded up, but the extra won't be usable, though it is put back on the free list if the used bit is freed.

The C RTL also keeps track of allocated and freed memory, since freed memory is rarely actually given back to the system. While the RTL tries to satisfy requests for new memory from the pool of freed chunks, this isn't always possible, and it's not too tough to accumulate a lot of little bits of memory that are too small to be of any use. The more alloc/free cycles you have, the more likely you are to have a significant fragmentation of your memory pool, which usually means lots of allocated but unusable memory.

A single big alloc with a resulting free can also blow your process' memory usage way out, even if you're no longer actually using the memory--it's on your process' free list, ready to be allocated and still counting against your memory size, but Devel::Size won't be able to find it.

And, of course, I may well have blown it somewhere. Wouldn't be the first time and, like I said, I'll go check and see. I'll also go update the readme for it so folks know what Devel::Size's numbers are good for, and what they aren't. A general set of docs on memory allocation is probably also in order.

In reply to Re: Re: Re: Memory Allocation by Elian
in thread Memory Allocation by Chief of Chaos

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.