in reply to memory usage and leakage

A few things to think about:

  1. Do you have a memory leak, or do you simply have memory consumption?   Test::Memory::Cycle is a good way to determine if you have self-referential data structures in a program.   (See also the other CPAN modules which it uses.)
  2. Is there memory pressure?   (If there isn’t, then the point-of-view of the always lazy operating system will be, “always room for one more!”   And, it is absolutely correct.)   The metrics produced by commands such as ps are only the roughest of indicators.   What you are really interested in is paging, particularly page steals, which is the actual indicator that something actually hurts.   If you don’t have pain, you don’t have a problem.
  3. Don’t guess ... measure.   If you suspect that excessive memory consumption is the root cause of your problem, demonstrate it.
  4. “Chips are cheap” now, whereas time is as expensive as ever.   I’m not talking about computer time here ... I’m talking about your own.   Sometimes the very best way to construct a program is to use the strongest software tools you can, then deal with any memory issues that actually develop (don’t borrow trouble ...) by “throwing silicon at it.”   After all, it’s only sand ...

Replies are listed 'Best First'.
Re^2: memory usage and leakage
by Sewi (Friar) on Jun 23, 2011 at 15:24 UTC

    For 2 & 4:

    You're right - for web or short term applications. Net::XMPP is used for IM connections and those are usually long-life. If there is any real memory leak (which increases the used memory over time even without additional data being stored), it will become a problem, sooner or later.
    Let the final script run for a hour or day and everything might be fine, but if it's logged into XMPP for weeks or even longer, memory usage will grow endless.
    The problem is even worse, because there are running multiple instances (dosn't matter if they are processes or threads).

    Restarting the whole application from time to time is bad, even if the new process re-logins to XMPP after some seconds. The problem will catch you sooner or later even if you try to reboot the app depending on memory usage.

    Try Padre - the free Perl IDE
[reply]
Re^2: memory usage and leakage
by szabgab (Priest) on Jun 26, 2011 at 10:32 UTC
    1. Sure, there was a leak. I could see it even without Test::Memory::Cycle but using that module helped me locate it and fix it. Thanks for pointing in it out.

    2. Sure there is a pressure. We made some simple calculations of how much memory we will need and it it was more than what the servers can handle.

    3. Wasn't my question exactly about how to measure?

[reply]

In Section Seekers of Perl Wisdom