Re^3: Calling a C function using malloc() in a XS

Thank you, Rob, for your participation and reproducing the code on your Win32/perl-5-12 environment. I described my problem bearing in mind perl 5.6 (I know, it's an old version ... maybe even an archaic one), but your perl version triggered my intention to try a more recent perl.

Firstly I wanted to try on a fresh environment. After reinstalling Win/MSVS6/Perl everything worked the same way ... Then I tried Perl 5.8 with the SAME (only #include "ppport.h" was added by h2xs) code and it worked correctly.

Maybe it's premature to state that it's a Perl 5.6 (more precisely ActiveState Perl 5.6.1 Build 638) bug, but Perl 5.8 doesn't have this memory leak problem.

Now I need to upgrade Perl/mod_perl. A last question would be what do you recommend for a 'not-enterprise production environment': 5.8, 5.10, or 5.12? Is there a stable mod_perl module built from 5.12 sources for Apache 2.2 (all Win32)? Or is it better to use mod_perl with Apache 2.0?

Comment on Re^3: Calling a C function using malloc() in a XS

Replies are listed 'Best First'.
Re^4: Calling a C function using malloc() in a XS by syphilis (Archbishop) on Aug 09, 2010 at 00:32 UTC
I described my problem bearing in mind perl 5.6 Aaah ... when I run my demo on perl-5.6 I can see what you mean - the memory usage does not drop down until the script exits. However, I think it's as ikegami says below: ...the memory is being put back into Perl's free memory pool and you just don't realize it. With perl-5.6 (running the script I posted), note that when do_simulation2() is called, the memory usage does not jump up. This is a sure sign that it is re-using the memory that was freed by do_simulation(). So there's no memory leaking going on, and no bug - it's just that with 5.6, freed memory stays locked up in the pool, whereas with 5.12 freed memory is being released back to the system. Task Manager therefore sees the memory release under 5.12, but doesn't see the memory release under 5.6. (It seems to me that 5.6's behaviour in this regard is probably more efficient than 5.12's ... but I expect that there are pros and cons.) Now I need to upgrade Perl/mod_perl. A last question would be what do you recommend ... Sorry - I'm way out of touch with Apache and mod_perl. I don't think you should be worrying about updating just on the strength of this memory handling behaviour of 5.6 ... unless, of course, you really do need the freed memory to be released back to the system. Cheers, Rob	[reply]
Re^5: Calling a C function using malloc() in a XS by ikegami (Patriarch) on Aug 09, 2010 at 01:36 UTC
So there's no memory leaking going on, and no bug - it's just that with 5.6, freed memory stays locked up in the pool, whereas with 5.12 freed memory is being released back to the system. That can vary based on which memory allocator was selected when Perl was built, and possibly based on the OS. The fact that the two versions of Perl are involved might be incidental.	[reply]
Re^5: Calling a C function using malloc() in a XS by Anonymous Monk on Aug 09, 2010 at 11:48 UTC
Yes, it seems not to be a bug, just a unconditioned feature. The memory does not drop down while the process is running and it's a very bad thing when used in a mod_perl environment. The interpreter is embedded into Apache server and the the former is permanently loaded in memory. So, after a first use of the do_simulation() functions, the process' memory grows up and remains at the highest limit. Sometimes this limit could be more than 1 GB Ok, perl 5.6 manages the memory this way. But why the (external) code provided in the XS file and using malloc()/free() has the same behaviour? Is it linked to a malloc() function provided by Perl itself? If so, how can I use the malloc() provided by my system? ("#undef malloc" doesn't work)	[reply]
Re^6: Calling a C function using malloc() in a XS by ikegami (Patriarch) on Aug 09, 2010 at 13:59 UTC
You could use an external process to do that job. A simulation that requires that much memory would probably benefit from being separate from the web server for other reasons anyway.	[reply]