in reply to Calling a C function using malloc() in a XS

The same problem with New() and Safefree():

#include "EXTERN.h" #include "perl.h" #include "XSUB.h" #include "stdlib.h" struct s { int field1; int field2; int field3; }; void do_simulation() { int i; struct s* m; New(1, (struct s*)m, 10000000, struct s); // do some computation with data // but for simplifying things I'll do a simple initialization for ( i=0; i<10000000; i++ ) { m[i].field1 = 0; m[i].field2 = 1; m[i].field3 = 2; } Safefree(m); } MODULE = MyModule PACKAGE = MyModule void sim() CODE: do_simulation();

Tested in a mod_perl environment and the memory continues to add up for each request (the final destination of this module will be the mod_perl environment)

Replies are listed 'Best First'.
Re^2: Calling a C function using malloc() in a XS
by syphilis (Archbishop) on Aug 08, 2010 at 13:12 UTC
    The same problem with New() and Safefree()

    I can't reproduce the problem using either malloc/free or New/safefree:
    use warnings; use Inline C => Config => BUILD_NOISY => 1; use Inline C => <<'EOC'; struct s { int field1; int field2; int field3; }; void do_simulation() { int i; struct s * m = (struct s*)malloc(10000000 * sizeof(struct s)); for (i=0; i<10000000; i++) { m[i].field1 = 0; m[i].field2 = 1; m[i].field3 = 2; } printf("Sleeping for 5\n"); sleep(5); free(m); printf("Memory freed ... sleep for another 5\n"); sleep(5); printf("Done\n"); } void do_simulation2() { int i; struct s* m; New(1, (struct s*)m, 10000000, struct s); for ( i=0; i<10000000; i++ ) { m[i].field1 = 0; m[i].field2 = 1; m[i].field3 = 2; } printf("Sleeping for 5\n"); sleep(5); Safefree(m); printf("Memory freed ... sleep for another 5\n"); sleep(5); printf("Done\n"); } EOC do_simulation(); do_simulation2();
    For me (perl-5.12.0, Win32) the memory is freed as soon as free/Safefree is called.

    Perhaps you're up against a feature of mod_perl ?

    Cheers,
    Rob

      Thank you, Rob, for your participation and reproducing the code on your Win32/perl-5-12 environment. I described my problem bearing in mind perl 5.6 (I know, it's an old version ... maybe even an archaic one), but your perl version triggered my intention to try a more recent perl.

      Firstly I wanted to try on a fresh environment. After reinstalling Win/MSVS6/Perl everything worked the same way ... Then I tried Perl 5.8 with the SAME (only #include "ppport.h" was added by h2xs) code and it worked correctly.

      Maybe it's premature to state that it's a Perl 5.6 (more precisely ActiveState Perl 5.6.1 Build 638) bug, but Perl 5.8 doesn't have this memory leak problem.

      Now I need to upgrade Perl/mod_perl. A last question would be what do you recommend for a 'not-enterprise production environment': 5.8, 5.10, or 5.12? Is there a stable mod_perl module built from 5.12 sources for Apache 2.2 (all Win32)? Or is it better to use mod_perl with Apache 2.0?

        I described my problem bearing in mind perl 5.6

        Aaah ... when I run my demo on perl-5.6 I can see what you mean - the memory usage does not drop down until the script exits.

        However, I think it's as ikegami says below: ...the memory is being put back into Perl's free memory pool and you just don't realize it.
        With perl-5.6 (running the script I posted), note that when do_simulation2() is called, the memory usage does *not* jump up. This is a sure sign that it is re-using the memory that was freed by do_simulation().

        So there's no memory leaking going on, and no bug - it's just that with 5.6, freed memory stays locked up in the pool, whereas with 5.12 freed memory is being released back to the system. Task Manager therefore sees the memory release under 5.12, but doesn't see the memory release under 5.6. (It seems to me that 5.6's behaviour in this regard is probably more efficient than 5.12's ... but I expect that there are pros and cons.)

        Now I need to upgrade Perl/mod_perl. A last question would be what do you recommend ...

        Sorry - I'm way out of touch with Apache and mod_perl. I don't think you should be worrying about updating just on the strength of this memory handling behaviour of 5.6 ... unless, of course, you really do need the freed memory to be released back to the system.

        Cheers,
        Rob
Re^2: Calling a C function using malloc() in a XS
by Anonymous Monk on Aug 08, 2010 at 12:06 UTC
    You have #include "stdlib.h" after perl includes, get rid of it

      I've tried without "stdlib.h". The same result for both cases (malloc()/free() and New()/Safefree())

      I aslo created a new MSVC6 project with the same code (perl paths included):

      #include "EXTERN.h" #include "perl.h" #include "XSUB.h" struct s { int field1; int field2; int field3; }; void do_simulation() { int i; struct s * m = (struct s*)malloc(10000000 * sizeof(struct s)); for (i=0; i<10000000; i++) { m[i].field1 = 0; m[i].field2 = 1; m[i].field3 = 2; } /* sleep for 5 seconds to be able to analyze memory usage sleep(5); free(m); /* sleep for 5 seconds to be able to see if memory is freed sleep(5); } int main(void) { do_simulation(); return 1; }

      ... and everithing works as it should: first 5 seconds the process uses 200+ MB (VM included), the last 5 seconds memory usage drops down to ~600 KB