Greets,

To catch memory leaks in the Perl bindings for Apache Lucy, we run the entire test suite under Valgrind. It takes a long time to complete, but it's an effective technique for catching problems in our C code.

However, Valgrind doesn't ordinarily catch leaks of Perl/XS data structures. As I understand things, Valgrind overrides malloc/calloc/realloc/free and installs its own versions (which track memory usage). However, if you have your own allocation scheme, as Perl does, Valgrind may not notice when things go awry.

Say that your custom allocator requests blocks in large chunks from the OS, and then frees them during a cleanup phase just prior to process exit. Valgrind will likely notice if you fail to free one of those large chunks -- but it won't notice if memory parceled out by your allocator leaks during ordinary operation. I believe that describes the situation with Perl.

The Test::LeakTrace module from CPAN has caught memory leaks for us that Valgrind has not -- for instance, a qr// structure created from Perl-space that our XS binding code leaked a refcount for. Theoretically, we might consider running our test suite under Test::LeakTrace, just as we run it under Valgrind, in order to perform a thorough check. However, what I would prefer to do is compile a custom debugging Perl without the custom allocator, so that allocation of Perl data structures goes through malloc/free/etc. and our existing usage of Valgrind can catch these leaks.

Looking at the file malloc.c in the Perl source (I'm using the tarball for 5.12.2), it seems that definining the NO_FANCY_MALLOC and PLAIN_MALLOC symbols might achieve this goal. 

  There are two macros which serve as bulk disablers of advanced
  features of this malloc: NO_FANCY_MALLOC, PLAIN_MALLOC (undef by
  default).  Look in the list of default values below to understand
  their exact effect.  Defining NO_FANCY_MALLOC returns malloc.c to th
+e
  state of the malloc in Perl 5.004.  Additionally defining PLAIN_MALL
+OC
  returns it to the state as of Perl 5.000.

[download]

I tried running ./Configure with those options: 

./Configure -DDEBUGGING -Dprefix=/usr/local/debugperl \
-DNO_FANCY_MALLOC -DPLAIN_MALLOC -es

[download]

It doesn't seem to have worked, so maybe I have to specify those symbols using something other than the command line.

Before I go try again, does our analysis seem sound and our approach seem promising? I haven't yet grokked the details of Perl's deep dark magic memory allocation.

In reply to Make Perl use real malloc by creamygoodness

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.