in reply to Managing C library memory in XS

To be quite honest, these approaches smell really bad to me, because they are basically threatening to put the stability of the interpreter environment at jeopardy by obliging you, in classic “C” style, to have to constantly muck-around with memory issues ... to “have to do things in exactly the right way, throughout your program, ‘or else.’ ”

To put it another way, you are obliging the Perl programming to know and to correctly account-for the present [memory ...] state of the API library, instead of being able to rely upon Perl-managed, garbage-collected objects as you are ordinarily able to do.   I would therefore design the API wrapper in a way that ruggedly eliminates this hazard.   In other words, don’t pass-around “pointers to” anything.   Have Perl allocate an ordinary string-type or hash-type variable and pour the values you need into it.   (Or, the reverse.)

If the Perl-side code needs to make persistent references to some C-library object, let your wrapper code maintain its own list of those objects and to assign a short random-but-unique “moniker” string to each.   These monikers are meaningless, are known only to the wrapper and to the Perl application, and are kept in a list maintained by the wrapper.   The wrapper provides the means by which C-library objects can be created and destroyed, and, through the mechanism of monikers, to make specific references to them, but the wrapper never exposes their addresses to anyone.   (If the Perl program fails to supply a valid moniker, say a nice Perl-world exception will be thrown by the wrapper.)   When a memory-address actually gets passed to the C-library, though, the wrapper is confident that this address, which is known only to itself, is “good,” because the wrapper does know the present memory-state of the API library.   The wrapper now has the ability to clean-up any lingering library-objects reliably when the Perl environment ends, because it has a complete list of them in its monikers-table.

In short, instead of very-dangerously exposing the C-library environment to the Perl environment, use the wrapper as a diplomat:   even though the C-world and the Perl-world do not natively speak with one another, the diplomat undertakes to provide a rugged environment that both of them can understand and which protects them from one another.   The diplomat is also the one that tries to wear the “acid-proof underwear,” absorbing any runtime exceptions that may be thrown and translating those exceptions, if they occur, into appropriate Perl-environment terms.

Replies are listed 'Best First'.
Re^2: Managing C library memory in XS
by petermogensen (Sexton) on May 05, 2014 at 15:13 UTC
    It seems you are not only questioning the hack I proposed using the NV field, but the entire idea of returning C objects to perl as blessed references to SVs holding the pointer value (in the case as an IV), correct? But isn't that a rather common approach? I just checked the JSON::XS library which IMHO is one of the best working CPAN modules. It goes even a step further. It returns an entire native C JSON struct byte-for-byte as a PV string ... and the typemap INPUTs it as the SvPVX pointer. - the (char *) pointer to the Perl string only cast as a JSON*. Isn't that also pointer pass-around and very-dangerously exposing the C-library environment to the Perl environment, ?

      Basically, “yes it is.”   And JSON::XS is a wonderful high-performance encoder/decoder for that reason.   But you really aren’t dealing with “multiple persistent internal library-states” and you really aren’t updating anything, either.   Furthermore, that package is intentionally all-about-speed.   The situation suggested to me in the OP appeared to be a little bit different; hence my suggestions for what I thought might work better in the OP’s specific case.   (And they are meant to be nothing more than that.)   There is noone way to do this sort of thing.”   The best way to “manage C-library memory in XS” really depends altogether upon the library in question.

        Speed is not completely irrelevant for the task at hand. That's antoher reason why I'm reluctant to try copy everything back and forth and like the idea of just blessing a reference to a pointer. Also, the C library actaully does have methods for almost everything, so there's very little API use for actually having wrapper Per blessed-hash objects. to manage stuff.

        But let me ask the question another way:

        What does the Perl interpreter guarantee about dual value objects? Is there any official guarantee of semantics?

        I Mean... Scalar::Util even advertises the use of dualvar() to set both an interger and a string value on a scalar. So some guarantees must be provided.

        If the guarantee is to alway copy and preserve the entire struct SV and struct xpvmg... and in general support multi-valued vars (PV,IV,NV) ...and.. hmm.. NV being the same size as IV, then there should be nothing wrong with the approach.