XS, pointer scope, strings, allocation responsibility, sv

patcat88 has asked for the wisdom of the Perl Monks concerning the following question:

In XS, when using sv_setpv or sv_setpvn, what happens to the char pointer? will perl create a SV around the pointer/string, and then free it eventually through normal GC? or will perl copy the contents/string that the pointer points to into its own new allocated memory range that makes up the SV and then its my responsibility to free the pointer manually or through the C block ending?

Similar question for SvPV, I get a pointer from perl, who is supposed to free this pointer? will perl GC/free it when the scalar goes out of scope? is this the pointer of the actual data inside the SV or a newly allocated memory range with the bytes copied from the SV to the new memory range? if I change the data at SvPV's pointer, will the SV be updated in perl land automatically?

What is the SVPVX pointer?

How is SvPV_set different from sv_setpv?

Does perl every "defragment" its SVs which would invalidated any pointers collected from SvPV or SvPVX?

How stable are pack()'s pointers? When does perl free or kill them? What happens if the scalar string in the scalar that pack gave a pointer to shrinks or expands, from Perl land, will the pointer from pack be still valid/the same?

Perl offers Newx and Safefree. How safe to interchange these 2 with native C library free and malloc? Can a pointer made with 1 be freed with the other or vice versa? More specifically on ActivePerl. Are there any cases when you want to use native C library memory management instead of perl memory management or vice versa?

Comment on XS, pointer scope, strings, allocation responsibility, sv_setpv, SvPV, pack's pointer

Replies are listed 'Best First'.
Re: XS, pointer scope, strings, allocation responsibility, sv_setpv, SvPV, pack's pointer by Anonymous Monk on Oct 24, 2010 at 10:11 UTC
Hopefull this will help you answer some of your questions perlapi sv_setpv Copies a string into an SV... sv_setpvn Copies a string into an SV... PerlGuts Illustrated define SvPVX `70 /* safe version of free */ 71 72 Free_t 73 safefree(Malloc_t where) 74 { 75 #ifdef DEBUGGING 76 if (debug & 128) 77 fprintf(stderr,"0x%lx: (%05d) free\n",(unsigned long)wher +e,an++); 78 #endif 79 free(where); 80 }` [download]	[reply] [d/l]
Re^2: XS, pointer scope, strings, allocation responsibility, sv_setpv, SvPV, pack's pointer by Tux (Canon) on Oct 24, 2010 at 10:50 UTC
The macro's can expand to other macro's that expand to other macro's ...: perl-current $ ./perl -Ilib Porting/expand-macro.pl SvPVX `sh cflags "optimize='-O2 -g'" try.c` -E try.c > try.i CCCMD = ccache g++ -DPERL_CORE -c -D_REENTRANT -D_GNU_SOURC +E -fPIC -DDEBUGGING -DDEBUGGING -fno-strict-aliasing -pipe -fstack-pr +otector -I/pro/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS= +64 -O2 -g -Wall -ansi -W -Wextra -Wendif-labels -Wwrite-strings -Wn +o-unused-variable -Wno-unused-parameter # 4 "SvPVX expands to" (({ SV const _svpvx = ((SV )({ void _p = (A0); _p; })); ((((svtype +)((_svpvx)->sv_flags & 0xff)) >= SVt_PV) ? static_cast<void> (0) : __ +assert_fail ("((svtype)((_svpvx)->sv_flags & 0xff)) >= SVt_PV", "SvPV +X expands to", 4, __PRETTY_FUNCTION__)); ((((svtype)((_svpvx)->sv_fla +gs & 0xff)) != SVt_PVAV) ? static_cast<void> (0) : __assert_fail ("(( +svtype)((_svpvx)->sv_flags & 0xff)) != SVt_PVAV", "SvPVX expands to", + 4, __PRETTY_FUNCTION__)); ((((svtype)((_svpvx)->sv_flags & 0xff)) != + SVt_PVHV) ? static_cast<void> (0) : __assert_fail ("((svtype)((_svpv +x)->sv_flags & 0xff)) != SVt_PVHV", "SvPVX expands to", 4, __PRETTY_F +UNCTION__)); ((!((((_svpvx)->sv_flags & (0x00004000\|0x00008000)) == 0 +x00008000) && (((svtype)((_svpvx)->sv_flags & 0xff)) == SVt_PVGV \|\| ( +(svtype)((_svpvx)->sv_flags & 0xff)) == SVt_PVLV))) ? static_cast<voi +d> (0) : __assert_fail ("!((((_svpvx)->sv_flags & (0x00004000\|0x00008 +000)) == 0x00008000) && (((svtype)((_svpvx)->sv_flags & 0xff)) == SVt +_PVGV \|\| ((svtype)((_svpvx)->sv_flags & 0xff)) == SVt_PVLV))", "SvPVX + expands to", 4, __PRETTY_FUNCTION__)); ((!(((svtype)((_svpvx)->sv_fl +ags & 0xff)) == SVt_PVIO && !(((XPVIO) (_svpvx)->sv_any)->xio_flags +& 64))) ? static_cast<void> (0) : __assert_fail ("!(((svtype)((_svpvx +)->sv_flags & 0xff)) == SVt_PVIO && !(((XPVIO) (_svpvx)->sv_any)->xi +o_flags & 64))", "SvPVX expands to", 4, __PRETTY_FUNCTION__)); &((_sv +pvx)->sv_u.svu_pv); })) [download] Sometimes that helps but sometimes it just obfuscates more then it explains Enjoy, Have FUN! H.Merijn	[reply] [d/l]
Re: XS, pointer scope, strings, allocation responsibility, sv_setpv, SvPV, pack's pointer by ikegami (Patriarch) on Oct 24, 2010 at 17:16 UTC
In XS, when using sv_setpv or sv_setpvn, what happens to the char pointer? Nothing. It is just read by those functions. will perl create a SV around the pointer/string, and then free it eventually through normal GC? No. (You wouldn't be able to do `sv_setpv(sv, "foo");` if it did since `"foo"` can't be freed.) or will perl copy the contents/string that the pointer points to into its own new allocated memory range Yes. "Copies a string into an SV." Similar question for SvPV, I get a pointer from perl, who is supposed to free this pointer? It still belongs to the SV. You're just looking at the SV. Don't free it. will perl GC/free it when the scalar goes out of scope? When the scalar's refcount reaches zero. is this the pointer of the actual data inside the SV or a newly allocated memory range with the bytes copied from the SV to the new memory range? The former. if I change the data at SvPV's pointer, will the SV be updated in perl land automatically? Yes. Note that the content of the string are UTF-8 encoded and must be UTF-8 encoded if `SvUTF8`. What is the SVPVX pointer? SvPV stringifies the scalar if required before returning the string buffer. SvPVX doesn't stringify, so you need to make sure there is a string buffer available to be obtained (using SvPOK) first. How is SvPV_set different from sv_setpv? One should use the macro version (`SvPV_set`). `SvPV_set` just sets the PV field. You need to do any string copy, upgrading and flag setting yourself. Does perl every "defragment" its SVs which would invalidated any pointers collected from SvPV or SvPVX? A lot of things can invalidate the pointer. If the scalar is freed, or if it's type is changed such that it no longer has a PV, or if it's string is freed, or if the buffer moves due to reallocation during growth, etc. You can't hold on to it for any length of time. What happens if the scalar string in the scalar that pack gave a pointer to shrinks or expands, from Perl land, will the pointer from pack be still valid/the same? No, not necessarily. Same as any other scalar. You'll need to copy the string or keep a reference to the scalar. (Don't forget to increase its ref count if you keep a pointer to the scalar.) Can a pointer made with 1 be freed with the other or vice versa? No.	[reply] [d/l] [select]