rodd has asked for the wisdom of the Perl Monks concerning the following question:

Respectable keepers of knowledge:

I'm working on binding some JS and Perl code using Bun FFI (https://bun.sh/docs/api/ffi) and perlembed. Things work good, except for XS dynamic loading. Probably due to the way the Bun FFI links to the boot_DynaLoader in my libperl.so, and the general lack of tools and knowledge on my part to circumvent, emulate or recode some of those parts.

So, my question: where or how in core is the libperl boot_DynaLoader C function defined? I just can't find it! I've been to the Perl core code and docs on XS, boot_DynaLoader and ExtUtils::Embed, but to my surprise I can't figure how it works or where its code resides in core (my core is 5.22.1).

Any leads, links or pointers to help me acquire the knowledge of the inner workings of dynamic loading would be of great help.

PD: I didn't want to bore you with my code, but here it is, for context: 

const xsInit = new JSCallback(
    (pTHX) => {
        Perl.Perl_newXS(
            perl,
            pstr('DynaLoader::boot_DynaLoader'),
            Perl.boot_DynaLoader.ptr,
            pstr(import.meta.file)
        );
    },
    { args: ['ptr'] }
);
const rc = Perl.perl_parse(perl, xsInit.ptr, argc, argv, envp);

[download]

The above does not dynamic loads XS modules as intended due to the way the FFI library exports symbols, which are meant primarily to be used within the JS realm, not to be sent back to libperl as a callback or (void *)() pointers. I've tried some other workarounds and practices, ie using my own compiled xs_init() function from C and FFI linking to that, but loading is still not working. I probably will be able to solve it at some point, but still, I'd like to understand the inner workings of the core in that regard. I got it working on darwin x86 by accident, but the same workaround fails in linux.

Replies are listed 'Best First'.
Re: boot_DynaLoader: what, where, how
by choroba (Cardinal) on Mar 23, 2024 at 16:07 UTC
    I don't see it in the source code, but once I build perl from the source, I see a file ext/DynaLoader/DynaLoader.c which contains its definition: 
    #ifdef __cplusplus
extern "C" {
#endif
XS_EXTERNAL(boot_DynaLoader); /* prototype to pass -Wmissing-prototype
+s */
XS_EXTERNAL(boot_DynaLoader)
{
#if PERL_VERSION_LE(5, 21, 5)
    dVAR; dXSARGS;
#else
    dVAR; dXSBOOTARGSXSAPIVERCHK;
#endif
#if PERL_VERSION_LE(5, 8, 999) /* PERL_VERSION_LT is 5.33+ */
    char* file = __FILE__;
#else
    const char* file = __FILE__;
#endif

    PERL_UNUSED_VAR(file);

    PERL_UNUSED_VAR(cv); /* -W */
    PERL_UNUSED_VAR(items); /* -W */
#if PERL_VERSION_LE(5, 21, 5)
    XS_VERSION_BOOTCHECK;
#  ifdef XS_APIVERSION_BOOTCHECK
    XS_APIVERSION_BOOTCHECK;
#  endif
#endif

        newXS_deffile("DynaLoader::dl_load_file", XS_DynaLoader_dl_loa
+d_file);
        newXS_deffile("DynaLoader::dl_unload_file", XS_DynaLoader_dl_u
+nload_file);
        newXS_deffile("DynaLoader::dl_find_symbol", XS_DynaLoader_dl_f
+ind_symbol);
        newXS_deffile("DynaLoader::dl_undef_symbols", XS_DynaLoader_dl
+_undef_symbols);
        newXS_deffile("DynaLoader::dl_install_xsub", XS_DynaLoader_dl_
+install_xsub);
        newXS_deffile("DynaLoader::dl_error", XS_DynaLoader_dl_error);
#if XSubPPtmpAAAA
        newXS_deffile("DynaLoader::CLONE", XS_DynaLoader_CLONE);
#endif

    /* Initialisation Section */

#line 171 "DynaLoader.xs"
    (void)dl_private_init(aTHX);

#if XSubPPtmpAAAA
#endif
#line 591 "DynaLoader.c"

    /* End of Initialisation Section */

#if PERL_VERSION_LE(5, 21, 5)
#  if PERL_VERSION_GE(5, 9, 0)
    if (PL_unitcheckav)
        call_list(PL_scopestack_ix, PL_unitcheckav);
#  endif
    XSRETURN_YES;
#else
    Perl_xs_boot_epilog(aTHX_ ax);
#endif
}

#ifdef __cplusplus
}
#endif

    [download]

    map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]
[reply]
[d/l]
[select]
Re: boot_DynaLoader: what, where, how
by hv (Prior) on Mar 23, 2024 at 16:48 UTC

    It appears to be constructed by ExtUtils::ParseXS, and weaves in the BOOT: section from the XS file. It is rather gnarly code, but in process_file() I think the work to emit it starts around the comment "# print initialization routine" (line 930 in my copy).

[reply]
[d/l]
[select]

      Ah, the classic "Perl code that writes C code that turns other C code into Perl functions". :-)

      I can hardly complain that "it's too convoluted". I spent the better part of a month writing code that takes a HTML Template Toolkit file, rips out the Javascript into it's own file, then rips any "onclick" (and similar) functions out of the remaining HTML and writes corresponding JQuery code into the new JS file, then throws both files through TT (including a DIY "translate and HTML quoting" plugin) and stuffs the results into an in-memory cache for future use...

      PerlMonks XP is useless? Not anymore: XPD - Do more with your PerlMonks XP
[reply]
        I'm pretty sure a PR implementing a compatible, less-"convoluted" way would be accepted!

        But I doubt such exists, because making a dynamic language out of C is quite complicated, especially when you want to allow people to interface eXternal Subroutines (XS for short), which Dynaloader is.

[reply]

Back to Seekers of Perl Wisdom