Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked

Re: Re: Re: Memory Allocation

by Elian (Parson)
on Feb 17, 2003 at 18:44 UTC ( #236066=note: print w/replies, xml ) Need Help??

in reply to Re: Re: Memory Allocation
in thread Memory Allocation

While it's possible I've missed something, and I'll go check, Devel::Size is generally an accurate representation of the amount of actual memory that perl uses for its internal data structures. This does not mean that it will give you an answer anywhere near what top will give you--far from it in some cases.

It can't, for example, show you the memory that might've been allocated by XS code and not managed by perl. (The expat XML library does this, as do the various DBD modules, for example) At some point I may put in support for a few known modules or libraries (like expat, which some of the XML modules use) but that's a dodgy thing, as it means Devel::Size needs grubby details of other people's internals as well as perl's.

It also doesn't show you memory your process (but not perl) may have allocated but that isn't currently tied up in a perl data structure. Memory allocations are generally done with a particular granularity, often a power of two or a factor of four or eight (depends on the type of request, OS, and processor architecture), and if your perl has requested a chunk of memory smaller than that amount it'll get silently rounded up, but the extra won't be usable, though it is put back on the free list if the used bit is freed.

The C RTL also keeps track of allocated and freed memory, since freed memory is rarely actually given back to the system. While the RTL tries to satisfy requests for new memory from the pool of freed chunks, this isn't always possible, and it's not too tough to accumulate a lot of little bits of memory that are too small to be of any use. The more alloc/free cycles you have, the more likely you are to have a significant fragmentation of your memory pool, which usually means lots of allocated but unusable memory.

A single big alloc with a resulting free can also blow your process' memory usage way out, even if you're no longer actually using the memory--it's on your process' free list, ready to be allocated and still counting against your memory size, but Devel::Size won't be able to find it.

And, of course, I may well have blown it somewhere. Wouldn't be the first time and, like I said, I'll go check and see. I'll also go update the readme for it so folks know what Devel::Size's numbers are good for, and what they aren't. A general set of docs on memory allocation is probably also in order.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://236066]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others romping around the Monastery: (2)
As of 2023-10-01 21:45 GMT
Find Nodes?
    Voting Booth?

    No recent polls found