Tying down how much memory a given program will use, and what if any of that memory will be recycled, either internally by perl or back to the OS, is extremely complicated. (As well as highly OS / perl -V / individual perl build depenedant.)

For example, these 2 one-liners

P:\test>perl -e" { for( 1 .. 100_000 ) { $x[ $_ ] = ' ' x 1000; $x[ $_ ] = undef; } <STDIN>; } <STDIN>;"

In this first example, each element of the 100_000 element global array @x is allocated a 1000-byte value, which is then immediately 'freed' by undefing it. At the end of the loop, (the first prompt), 100+MB is allocated to the process. The space for 100_000 elements of 1000-bytes + the overhead for perls array and scalar structures. Even though only 1 element of the array has any space allocated (usable) at any given time.

P:\test>perl -e" { my @x; for( 1 .. 100_000 ) { $x[ $_ ] = ' ' x 1000; $x[ $_ ] = undef; } <STDIN>; } <STDIN>;"

The same program, except that the array is now locally scoped. When the first prompt is reached after the loop completes, again, 100+MB is being used, meaning that 99_999 elements of discarded (undef'd) space are lying around unusable and unused. However, once the second prompt is reached, ie. after the local scope in which @x was defined has exited, the memory used by the process (on my system) drops to 12MB.

With care and motivation, it is possible to force perl to re-use discarded memory, (and even return some of it to the OS under win32), but every attempt I've made to formulate a strategy for doing either, has fallen on stoney ground. I can do it on a case-by-case basis for many apps. I have begun to recognise some cases where I am reasonably sure that I can optimise the memory requirements through fairly simple steps, but inevitably, there are always exceptions to the rules of thumb I use.

Unfortunately, the exceptions are too common to make the rules of thumb viable for anything other than cases of extreme need.


Examine what is said, not who speaks.
"Efficiency is intelligent laziness." -David Dunham
"Think for yourself!" - Abigail

In reply to Re: Re: Re: Memory usage breakup by BrowserUk
in thread Memory usage breakup by eXile

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.