BrowserUk has asked for the wisdom of the Perl Monks concerning the following question:

Does anyone know where the title file comes from?

I'm attempting to build Data::Alias on Win32. The pod warns:

Platform support

Some platforms, notably Windows and AIX, will give link errors when attempting to build Data::Alias. This cannot be fixed. When using Windows, you can use perl inside cygwin (http://www.cygwin.com/) instead, where Data::Alias does work.

And sure enough, the link fails for the lack of the ability to resolve the following symbols:

Alias.obj : error LNK2019: unresolved external symbol ... _Perl_av_reify referenced in function _da_refgen _Perl_vivify_defelem referenced in function _da_refgen __imp__PL_no_localize_ref _Perl_pp_rv2av referenced in function _DataAlias_pp_rv2av _Perl_pp_rv2hv referenced in function _DataAlias_pp_rv2hv __imp__PL_sv_placeholder _Perl_pop_return referenced in function _DataAlias_pp_return _Perl_setdefout referenced in function _DataAlias_pp_return _Perl_lex_end referenced in function _DataAlias_pp_return _Perl_pp_entereval referenced in function _DataAlias_pp_entereval _Perl_mod referenced in function _da_transform _Perl_qerror referenced in function _da_lvalue __imp__PL_op_mutex _Perl_yylex referenced in function _da_ck_rv2cv _Perl_op_clear referenced in function _da_ck_entersub blib\arch\auto\Data\Alias\Alias.dll : fatal error LNK1120: 15 unresolv +ed externals

The leading underscore on those symbol names (maybe) indicates that this is the familiar problem of the default calling convention is being used. The (maybe) fix being to tag those functions to use the pascal calling convention.

Whilst grepping around for the locations of the declarations, I noticed that several of them turn up in the title file, and that file contains:

embed.fnc embed.h global.sym op.c perly.c perly.h perly.y perly_c.diff proto.h toke.c vms/perly_c.vms vms/perly_h.vms

which look like the right places to be fixing the declarations. If it is, then it means that this is (at least in part) a solved problem. I'm hoping that by inspecting the build files for whatever module uses it I can work out how these files are used. But for the life of me, a cannot work out where this file came from? Neither a cpan search nor google have turned up anything. Is there a better way?


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

Replies are listed 'Best First'.
Re: From whence comes perl-5.8.6-smartmatch.tar?
by syphilis (Archbishop) on Dec 22, 2006 at 02:42 UTC
    I once asked a question to the xs mailing list (http://www.mail-archive.com/perl-xs@perl.org/msg02074.html) as to why the 'Perl_report_uninit' function produced an 'undefined reference' error on Windows, but worked fine on Linux. Jan Dubois provided the following:

    --- quote ---
    "..Perl_report_uninit() is not a public Perl API, so extensions are not allowed to call it. On Linux this cannot be enforced as all internal symbols are being exported anyways, but on Windows (and AIX) you can only call APIs that are marked as "public" in the embed.fnc file in the core Perl distribution.

    In general it is a good idea to check `perldoc perlapi`. If the function isn't listed in there, then you are not allowed to use them in an extension...."
    --- end quote ---

    (I'm sure I've used functions in an extension that are not listed in perlapi - though the advice offered in Jan's second paragragh is most likely sound advice.)
    I suspect that you've come up against a similar issue. In my case, afaict, 'Perl_report_uninit' was simply not there in libperl58.a (or libperl58.lib, as the case may be) - and I decided that this was going to be much too difficult to work around (especially given that I had no compelling incentive to do so).

    Hope this helps. If you have some ideas as to how this difficulty might best be overcome, please share them, as I'd be interested to learn a little more about this.

    Cheers,
    Rob
      If you have some ideas as to how this difficulty might best be overcome, please share them,

      There would appear to be 3 possible ways forward:

      1. Persuade the p5p guys to export these entrypoints on win32/AIX etc.

        As their privacy cannot be enforced on the defacto base development platform, and module authors are finding these functions useful(*), only protecting them on platforms that have that ability almost guarantees that those platforms will be seen as the "awkward squad" for failing to build modules that "work just fine on unix".

        (*)I found several other modules on my system that contain references to av_reify: DBI (does build); Devel::FastProf (doesn't); Inline (does); Storable (does); threads (does); Time::HiRes (does); (My own) Win32::Fmode! (which does).

        In the latter case, the only reference is in ppport.h, which is automatically generated/incorporated by the "build & packaging process" and I do not pretend to understand. The same may be true of all the other packages that do build. Though DBI seems to be different?

      2. Build a dll that contains (copies) of all the non-api functions visible to linux authors for use in building modules that *need* them.

        A recipe for disaster if ever I heard one, but no more so than the other option randyk mentioned. Ie. C&Ping the source code into the individual modules. At least if a single 'compatibility' module/DLL was available, there would only be one place to try and keep in sync with the P5 sources.

      3. Accept the status quo.

        Accept that there will be a bunch of modules that will build on linux--because although the authors are playing fast and loose with the 'rules' of what internal APis they use, there is no way to enforce those rules--that will not build on other platforms where those rules can be enforced.

        Kinda reminds me of the state of play with the UK and EU. The UK rigidly enforces EU rules, even those that it voted against, on it's own citizens, even though other EU states (often those that voted for or even vetoed the legislation into existence), quietly allow their citizens to flout or just outright ignore those rules.

        It's also remeniscent of another debate between sauoq and others elsewhere on PM. Basically, it's better to be very careful about what laws you put on the books in the first place, than to try and deal with the gamut of loopholes, workarounds and bypasses that will ensue when bad legislation gets through for the lack of challenge.

      Seems that there is scope for a reassessment of the decision making process that choses which APIs are public and which not. That would inevitably need to include discussion of why those authors that are using private APIs in their extensions are doing so, and is there any alternative. It might also prompt a discussion of moving to a 'permissive by default' stance that would expose all internal APIs except where there was specific reason for not doing so.

      However, I can well understand that the p5p guys would view any such discussion with skepticism, if not outright hostility. It's a fairly common rule to restrict the exposure of an API to the minimum commensurate with functionality, and I foresee considerable resistance to attempting to change that. Were I on the other side of the debate, I might find myself in that camp also.

      The bottom line is that for the vast majority of Win32 Perl users that choose to use binary builds, there will always be some subset of modules (usually the more useful ones!), that they will continue to be prevented from using by circumstances beyond their control. If the authors of these extensions are finding that they need to ignore the public/private boundaries of the Perl API in order to get what they are doing done, it tends to indicate there is something wrong with the positioning of the boundaries--but that's a discussion I am unqualified to lead.

      Other possibilities--like every individual using Cygwin; or AS patching embed.fnc--are simply spreading the 'EU states ethic' that you simply ignore the rules that you dislike.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
        If the authors of these extensions are finding that they need to ignore the public/private boundaries of the Perl API ...

        In the case I had earlier come across, I considered it possible that the author was unaware of the problem (and further pondered whether he did, in fact, need to use 'Perl_report_uninit').

        It seems likely, however, that the Data::Alias author is very much aware of the situation, and sees no way to avoid using the "forbidden" functions.

        .... spreading the 'EU states ethic' that you simply ignore the rules that you dislike

        I like the metaphor :-)

        Cheers,
        Rob
Re: From whence comes perl-5.8.6-smartmatch.tar?
by randyk (Parson) on Dec 22, 2006 at 05:29 UTC
    In some cases, it may be possible to just paste in the source code into the xs file for the missing functions; for this, http://b-src.cbrc.jp/ is quite useful for searching through the Perl sources. However, as Rob warns, there may be good reasons why such functions are not part of the public API.
Re: From whence comes perl-5.8.6-smartmatch.tar?
by sgt (Deacon) on Dec 22, 2006 at 14:33 UTC

    Yes, a sad state of affairs indeed. I came across Data::Alias looking at Elizabeth Mattijsen's Data::Reuse, but I remember problems building Data::Alias on HP-UX 11.0, an then did not even try cygwin.

    Maybe Data::Bind could be used in some places. On the positive side direct support from the core for true aliases is planned; Nicholas Clark has actually already done a lot (IIRC nobody commented on p5p, but true aliases are needed for the perl6_on_perl5 scheme).

    cheers --stephan
Re: From whence comes perl-5.8.6-smartmatch.tar?
by xmath (Hermit) on Jan 09, 2007 at 16:01 UTC
    As syphilis mentioned, the link errors are because Data::Alias digs a fair bit into perl's guts to do its job. It meddles with lexer state, modifies the PL_check[] array, and do various other things that are no doubt considered big "no-no"s. :-)

    Work-arounds are using cygwin or recompiling perl with an augmented list of exported symbols.

    I've also been pondering if there's some rude way for me to manually resolve those symbols are runtime, even if they're not exported. I'd need to dig a bit into what object formats and linkage are like on Win32.

      I've also been pondering if there's some rude way for me to manually resolve those symbols are runtime, even if they're not exported.

      The simple answer is no. Unfortunately, the complicated answer is no also ;(

      There really is no way to know where a piece of code has been stuffed in a dll if it's entrypoint is not exported.

      The real answer here is to export all the entrypoints. It costs (almost) nothing to do and so what if people can call entrypoints that are not documented as public? If those entrypoints go away or change, and their code breaks--it's their problem.

      Given that there isn't anyway to police this on most platforms including the 'reference platform', it's very much at odds with Perl's usual permissive stance, and very unfair on user of those few platforms that can enforce it.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
        There really is no way to know where a piece of code has been stuffed in a dll if it's entrypoint is not exported.
        No debugging symbols or whatever?

        The real answer here is to export all the entrypoints.
        I agree