Re: Perl, C and deallocation responsibility

There is a typedef mechanism that translates standard C types into an appropriate Perl type. I haven't used that myself. I have only written XS code using the Perl types directly. I find it less confusing as to who is doing what in terms of memory management. You will need to use Perl functions to allocate memory if you expect Perl to manage that memory. Of course, Perl will never release any memory back to the O/S, but it will reuse it for its own purposes!

See Inline Cookbook Pod for the greeting() program (shown below). This code works on my machine and discusses memory management. I would use SV types for input to your proc_desc() subroutine and manipulate them appropriately. Perhaps allowing for something more complex than just an 8 bit ASCII string (UTF-8?).

I think you will wind up with 2 input SV's and one output SV: SV* proc_desc(SV* a, SV* b) { ....code... }

use strict;
use warnings;
use Inline 'C';

print greeting('Ingy');

__END__

See https://metacpan.org/dist/Inline-C/view/lib/Inline/C/Cookbook.pod
and the example for "greeting" subroutine.

"I would urge you to stay away from mallocing your own buffer. Just us
+e Perl's built in
memory management. In other words, just create a new Perl string scala
+r. The function
newSVpv does just that. And newSVpvf includes sprintf functionality.

The other problem is getting rid of this new scalar. How will the ref 
+count get
decremented after we pass the scalar back? Perl also provides a functi
+on called
sv_2mortal. Mortal variables die when the context goes out of scope. I
+n other words, Perl
will wait until the new scalar gets passed back and then decrement the
+ ref count for you,
thereby making it eligible for garbage collection. See perldoc perlgut
+s.

In this example the sv_2mortal call gets done under the hood by XS, be
+cause we declared
the return type to be SV*."

__C__

SV* greeting(SV* sv_name) {
  return (newSVpvf("Hello %s!\n", SvPV(sv_name, PL_na)));
}
[download]

Comment on Re: Perl, C and deallocation responsibility Download Code

Replies are listed 'Best First'.
Re^2: Perl, C and deallocation responsibility (Returning Memory back to the OS References) by eyepopslikeamosquito (Archbishop) on Apr 29, 2023 at 07:20 UTC
> Of course, Perl will never release any memory back to the O/S, but it will reuse it for its own purposes! This is not correct. It depends on the implementation of the user-level malloc function. For more detail, see: Re: Not able to release memory (malloc implementation) which notes that on Linux and Windows at least (unlike most other Perl platforms) the standard C Library `malloc` function does return "big" chunks of memory back to the OS. Update: Though it's possible for a Perl built with a suitable `malloc` function to return memory to the OS, in practice this happens rarely, as described in ikegami's excellent Mini-Tutorial: Perl's Memory Management - which concludes with "You can't rely on memory being returned to the system, but it can happen ... if and when memory is returned to the OS is dependant on the memory allocation mechanism perl was compiled to use ... you are more likely to see memory being released to the OS on Windows". See also: perlfaq3: How can I make my Perl program take less memory perlfaq3: How can I free an array or hash so my program shrinks Re: Re: Re: Clearing an Array of Array's from memory by Elian (whether the freed space is returned to the operating system is OS, OS version, C library, and allocation size dependent) Re^3: Memory usage double expected (run-time) (notes from Tim Bunce's excellent Profiling Memory Usage talk - with examples showing why Perl sometimes surprises by not returning memory to the OS) Is malloc part of the kernel or the C Library? (SO) - malloc is part of the standard C Library, not the OS kernel malloc_trim (Linux glibc-specific function to force malloc to free all eligible OS pages to the kernel) What causes Ruby memory bloat? (blog post describing `malloc_trim`) References Added Later How to implement a Queue that doesn't leak memory? by anonymonk (2024) Updated: minor improvements to wording; added extra references.	[reply] [d/l] [select]
Re^3: Perl, C and deallocation responsibility by Marshall (Canon) on Apr 30, 2023 at 07:11 UTC
Thanks for the info! I haven't seen memory go back to the O/S, but I haven't had the need to look closely on Win 10. The last time I got really anxious about memory was when developing a Perl application for Win XP. I had a memory budget for that one. Often for Perl applications, I see memory usage approach a limit in an asymptotic way and then just stay flat forever. Of course, mileage varies a lot depending on the application's needs. I think in the context of the OP's question, one question is whether anybody is going to do anything to either free or reuse the memory allocated for the char. I was happy to have found an explicit statement about what is going to happen if the listed memory allocation routine is used. I was and am not at all clear what actually happens with the char declaration (instead of an SV that Perl will reference count upon). There is obviously some type mangling that happens underneath the covers, but I'm not sure how. BTW, I have found that SQLite is very good at returning memory if you adjust its memory usage during run time. Giving an SQL indexing operation a lot of memory can have a huge positive performance effect. And then shrink down memory size for normal usage.	[reply]
Re^2: Perl, C and deallocation responsibility by SheWolf (Novice) on Apr 30, 2023 at 23:02 UTC
Thank you. I hadn't thought of doing that. I think I was stuck on the whole 'must be a PV' thing. I'll try to play with that when I can!	[reply]
Re^3: Perl, C and deallocation responsibility by etj (Priest) on May 02, 2023 at 13:38 UTC
I would say what's most relevant in your decision is how Perl-dependent you want your C code to be: If you're comfortable with it being Perl-dependent, you would probably want to have all memory allocation/management done by Perl, using probably-mortal SVs, and PVs from the start. If you want to maintain the C code as purely C, then you can use the `typemap` stuff in the various notes here to copy the C-created data into Perl-land, and then `free` the C-allocated memory.	[reply] [d/l] [select]
Re^4: Perl, C and deallocation responsibility by SheWolf (Novice) on May 03, 2023 at 00:38 UTC
Unfortunately I need to make this work in both worlds. I still need to keep the compiled C running, and I need to have it called by Perl. I never choose the easy path!	[reply]