Re^4: Calling a C function using malloc() in a XS

I described my problem bearing in mind perl 5.6

Aaah ... when I run my demo on perl-5.6 I can see what you mean - the memory usage does not drop down until the script exits.

However, I think it's as ikegami says below: ...the memory is being put back into Perl's free memory pool and you just don't realize it.
With perl-5.6 (running the script I posted), note that when do_simulation2() is called, the memory usage does *not* jump up. This is a sure sign that it is re-using the memory that was freed by do_simulation().

So there's no memory leaking going on, and no bug - it's just that with 5.6, freed memory stays locked up in the pool, whereas with 5.12 freed memory is being released back to the system. Task Manager therefore sees the memory release under 5.12, but doesn't see the memory release under 5.6. (It seems to me that 5.6's behaviour in this regard is probably more efficient than 5.12's ... but I expect that there are pros and cons.)

Now I need to upgrade Perl/mod_perl. A last question would be what do you recommend ...

Sorry - I'm way out of touch with Apache and mod_perl. I don't think you should be worrying about updating just on the strength of this memory handling behaviour of 5.6 ... unless, of course, you really do need the freed memory to be released back to the system.

Cheers,
Rob

Comment on Re^4: Calling a C function using malloc() in a XS

Replies are listed 'Best First'.
Re^5: Calling a C function using malloc() in a XS by ikegami (Patriarch) on Aug 09, 2010 at 01:36 UTC
So there's no memory leaking going on, and no bug - it's just that with 5.6, freed memory stays locked up in the pool, whereas with 5.12 freed memory is being released back to the system. That can vary based on which memory allocator was selected when Perl was built, and possibly based on the OS. The fact that the two versions of Perl are involved might be incidental.	[reply]
Re^5: Calling a C function using malloc() in a XS by Anonymous Monk on Aug 09, 2010 at 11:48 UTC
Yes, it seems not to be a bug, just a unconditioned feature. The memory does not drop down while the process is running and it's a very bad thing when used in a mod_perl environment. The interpreter is embedded into Apache server and the the former is permanently loaded in memory. So, after a first use of the do_simulation() functions, the process' memory grows up and remains at the highest limit. Sometimes this limit could be more than 1 GB Ok, perl 5.6 manages the memory this way. But why the (external) code provided in the XS file and using malloc()/free() has the same behaviour? Is it linked to a malloc() function provided by Perl itself? If so, how can I use the malloc() provided by my system? ("#undef malloc" doesn't work)	[reply]
Re^6: Calling a C function using malloc() in a XS by ikegami (Patriarch) on Aug 09, 2010 at 13:59 UTC
You could use an external process to do that job. A simulation that requires that much memory would probably benefit from being separate from the web server for other reasons anyway.	[reply]