Memory leak in XS code that handles C++ exceptions

slower has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks,

I'm having trouble with a memory leak in XS code that occurs whenever I catch a C++ exception, then croak. I would appreciate your advice as to whether this is a problem with my XS code, or with Perl itself.

I can reproduce the problem in a trivial test library, libfoo:

foo.h ------------------------------------------------------
class Foo {
  public:
    void bar();
};
[download]

foo.cpp ----------------------------------------------------
#include <stdexcept>
#include "foo.h"

void Foo::bar() {
  throw std::runtime_error("oh no!");
}
[download]

Here is the XS glue:

Foo.xs -----------------------------------------------------
void
Foo::bar()
  CODE:
    try {
      THIS->bar();
    }
    catch (const std::runtime_error &e) {
      croak("Foo::bar threw a runtime error!");
    }
[download]

My Perl test script just instantiates Foo and calls bar() in an eval block n times. For n=1000, valgrind reports the following on RHEL5, x64, Perl 5.12.2:

30,969 bytes in 999 blocks are possibly lost in loss record 1,064 of 1
+,069
   at 0x4A0666E: operator new(unsigned long) (vg_replace_malloc.c:220)
   by 0x929F070: std::string::_Rep::_S_create(unsigned long, unsigned 
+long, std::allocator<char> const&) (new_allocator.h:89)
   by 0x929FCA4: char* std::string::_S_construct<char const*>(char con
+st*, char const*, std::allocator<char> const&, std::forward_iterator_
+tag) (basic_string.tcc:139)
   by 0x929FE41: std::basic_string<char, std::char_traits<char>, std::
+allocator<char> >::basic_string(char const*, std::allocator<char> con
+st&) (basic_string.h:1546)
   by 0x8FFEABF: Foo::bar() (foo.cpp:6)
   by 0x8DFC856: XS_Foo_bar (in /home/slower/dev/xstest/src/blib/arch/
+auto/Foo/Foo.so)
   by 0x49291F: Perl_pp_entersub (in /usr/local/perl/5.12.2-gcc443-rhe
+l5-64/bin/perl)
   by 0x490D35: Perl_runops_standard (in /usr/local/perl/5.12.2-gcc443
+-rhel5-64/bin/perl)
   by 0x43410B: perl_run (in /usr/local/perl/5.12.2-gcc443-rhel5-64/bi
+n/perl)
   by 0x41E51B: main (in /usr/local/perl/5.12.2-gcc443-rhel5-64/bin/pe
+rl)

143,856 bytes in 999 blocks are possibly lost in loss record 1,068 of 
+1,069
   at 0x4A05E1C: malloc (vg_replace_malloc.c:195)
   by 0x92BF206: __cxa_allocate_exception (eh_alloc.cc:102)
   by 0x8FFEACD: Foo::bar() (foo.cpp:6)
   by 0x8DFC856: XS_Foo_bar (in /home/slower/dev/xstest/src/blib/arch/
+auto/Foo/Foo.so)
   by 0x49291F: Perl_pp_entersub (in /usr/local/perl/5.12.2-gcc443-rhe
+l5-64/bin/perl)
   by 0x490D35: Perl_runops_standard (in /usr/local/perl/5.12.2-gcc443
+-rhel5-64/bin/perl)
   by 0x43410B: perl_run (in /usr/local/perl/5.12.2-gcc443-rhel5-64/bi
+n/perl)
   by 0x41E51B: main (in /usr/local/perl/5.12.2-gcc443-rhel5-64/bin/pe
+rl)
[download]

These leaks completely disappear if either 1) Foo::bar doesn't throw an exception, or 2) Foo.xs catches the exception and doesn't croak. I have also confirmed the leak on the following platforms, though valgrind isn't available or doesn't detect it. Watching the memory usage of the test script reveals the leak in these cases (usage increases linearly with the value of n):

Solaris 8, x32, Perl 5.10.0
RHEL5, x32, Perl 5.10.0
Windows 7, x32, Perl 5.12.2
Windows 7, x64, Perl 5.12.2

The only platform available to me where the leak does not occur is Windows 7, x32, Perl 5.10.0.

Thanks,
m.

Comment on Memory leak in XS code that handles C++ exceptions Select or Download Code

Replies are listed 'Best First'.
Re: Memory leak in XS code that handles C++ exceptions by davido (Cardinal) on Oct 03, 2013 at 00:36 UTC
I was hoping to have seen an answer to this by now, but as we haven't, you might want to hit the perl-xs@perl.org mailing list over at http://lists.perl.org. Dave	[reply]
Re: Memory leak in XS code that handles C++ exceptions by bulk88 (Priest) on Oct 03, 2013 at 02:31 UTC
Does your unix longjmp() call C++ destructors (some do, see http://msdn.microsoft.com/en-us/library/yz2ez4as%28v=vs.90%29.aspx? croak is implemented with longjmp. Would you call longjmp from a catch block if this wasnt Perl? The simplest longjmp implementation will make the C++ exception dispatcher leak all of its resources. WAG, do a goto from the catch block to the parent block (outside of try/catch), then call croak. Or try a finally block.	[reply]
Re^2: Memory leak in XS code that handles C++ exceptions by slower (Acolyte) on Oct 03, 2013 at 14:53 UTC
Thanks bulk88, I wasn't aware of the internal implementation of croak, or longjmp for that matter. I see that longjmp combined with C++ objects can be a Very Bad Thing: No destructors for automatic objects are called. If replacing of longjmp with throw and setjmp with catch would execute a non-trivial destructor for any automatic object, the behavior of such longjmp is undefined. From http://en.cppreference.com/w/c/program/longjmp Your suggestion to croak outside the catch block worked! I actually don't even need a goto; I can just capture the exception message in a string, let the catch block exit normally, then croak with the captured message like so: void Foo::bar() CODE: const char* error = 0; try { THIS->bar(); } catch (const std::runtime_error &e) { error = e.what(); } if (error != 0) { croak(error); }	[reply]
Re: Memory leak in XS code that handles C++ exceptions by Anonymous Monk on Oct 03, 2013 at 08:47 UTC
See also ExtUtils::XSpp::Exception - Map C++ exceptions to Perl exceptions	[reply]
Re: Memory leak in XS code that handles C++ exceptions by Anonymous Monk on Oct 03, 2013 at 01:27 UTC
Here is the XS glue: Hi, I don't XS or CPP everyday, but if you posted something more complete, something self-contained like this super short example SOso-0.01.patch.txt, I would take a look	[reply]