in reply to module memory usage

This is from the ModPerl Cookbook, pulled from pages 317-322 (i'm quoting mixed with some paraphrasing).

The B::Size and B::TerseSize modules, both part of the B::Size distribution, include a number of routines to calculate memory usage of perl packages, subroutines and variables.

It goes on to describe how to set it up with Apache::Status on a mod_perl server to get useful information displayed by Apache::Status. If you would like more info on that, msg me and I will give it to you. But if you are not running mod_perl, then i would just look at the B::Size documentation. I cannot speak for it, as i have never used or known of anyone who has used it, but this Cookbook has never let me down before, so I weigh its suggestions pretty highly.

The book then goes on to talk about reducing memory consumption by not exporting symbols from modules, basically doing this: 


use POSIX ();

# instead of this ...

use POSIX;

[download]

POSIX imports over 560 symbols, each imported symbol being about a 120 byte memory penalty. According to this book that works out to 140KB of memery overhead that could be avoided with 2 colons and a little more typing. Not alot in a small one-off script, but that piles up fast when you have 50-100 mod_perl/Apache child processes screaming for attention from the CPU.

But as anyone who has been around in this biz for over 18 months (the cycle time for Moore's Law) will tell you. Don't worry about stuff like this too much unless you know for absolutely sure its a problem, because chances are in another 18 months,... well you know.

-stvn

Replies are listed 'Best First'.
Re: Re: module memory usage
by diotalevi (Canon) on Dec 22, 2003 at 06:02 UTC
    POSIX imports over 560 symbols, each imported symbol being about a 120 byte memory penalty. According to this book that works out to 140KB of memery overhead that could be avoided with 2 colons and a little more typing. Not alot in a small one-off script, but that piles up fast when you have 50-100 mod_perl/Apache child processes screaming for attention from the CPU.

    You'll also want to realize that the qualifiied name adds a handful of additional dereferences and a hash lookup when you could have spent the memory on a local access.

    --
  Devil
[reply]
      Devil,

      Quite true, the classic CPU versus memory tradeoff. As with all "optimizations", you almost never get away for free.

      Personally I would rather waste a CPU cycle or 2 since they tend to be very cheap, and for what I do (mod_perl web apps) a pico second here or there is usually okay, while running into disk swap for memory is not.

      In the end, I stick with the old mantra:

      Premature optimization is the route of all evil.

      - C.A. Hoare (although usually attributed to Donald Knuth)

      -stvn
[reply]
        Nah, this stuff is in the realm of micro-optimizations. That's why I signed my note with 'Devil' for 'Devil's advocate'.
[reply]
      What kind of genius has 50-100 mod_perl/Apache child processes?
[reply]

        I recently built a site which uses mod_perl to segment content to different user groups, and has a CMS backend. At first I had the MaxClients set to 50, and that turned out to be not nearly enough, from there we moved it to 100, which also wasn't enough. We settled on 125.

        Of course this does not mean that there are always 125 child processes serving 125 concurrent requests. But on average, there are between 50-100 processes running at any given time serving concurrent requests. The site regularly gets anywhere between 900-1400 unique homepage hits per day, most of which are concetrated between 8 a.m. and noon. And being an intranet site, it is actually read/used by its users. Most sessions run approx. 20 minutes to an hour.

        If I could have, I would have load balanced it, yadda yadda yadda, but economics and politics dictated that it had to run on a single dedicated Linux box with a shitload of RAM (I wasn't even allowed to put the MySQL DB on its own machine, it shares the same box).

        The site has had no issues, its response time is excellent, and there has been no complaints at all. Take note that this was a transparent replacement of an existing static HTML site, so users expectations on performance were pretty high.

        While I was pretty happy with the results, I would not call myself a genius.

        ;-)

        -stvn

        UPDATE: Oops, the site is actually running on FreeBSD, it was late last night when I posted this. I like Linux, but i gotta give credit where credit is due. (FreeBSD 4.7-RELEASE-p22 to be exact)

[reply]

In Section Seekers of Perl Wisdom