Exec Summary
------------
I'm struggling with memory management in long running perl scripts (daemons) the lifetime of these scripts are measured in days. The scripts run on servers running Linux.

Background
----------
For complicated reasons I'm stuck on perl 5.8.8 and linux kernel 2.4.31. The scripts are Event based daemons that use timer and io Event callbacks. There are few if any XS based modules in use (besides those in the core perl distrubution). Inside of the callbacks I allocate anonymous array and hashes that are used to pass data to the depths of the script. I utilize Data::Dumper and eval() to persist some of the structures to disk so the scripts can pick up where they left off after a restart. The scripts continue to consume memory over time until finally they have to be restarted to free up memory.

Troubleshooting so far
----------------------
I've used Devel::Leak, Devel::Mallinfo, Devel::Cycle to try to get a better understanding of the problem. The mallinfo shows that there is plenty of free memory in the process, but yet the process continues to allocated more memory (ie hints at memory fragmentation). Devel::Leak shows that there are allocations that are not being freed, Devel::Cycle does not show any internal cycles in the array/hash refs.

Questions
---------
0) What additional information can I provide to help others help me?
1) Are there any known issues with perl 5.8.8 that would lend itself to this issue?
2) Are there known issues with versions of Event that would cause this issue?
3) Is there a fundamental difference in how perl allocates memory for anonymous arrays/hashes vs @arrays and %hashes? (ie stack vs heap?) that would affect memory management?
4) Can the use of eval() cause this sort of issue?
5) Are there different 'tools' I could use to dig deeper?
6) Are there any red flags in my description that are 'bad practices' for these types of scripts?

Thank you for your help

In reply to Memory management with long running scripts by jamesrleu

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.