DrWhy has asked for the wisdom of the Perl Monks concerning the following question:

I've been working on my first XS extension, a C++ extension that calculates relative quality of different translations. After some frustrations and false starts it appears to be working fine now. It only compiles with MSVC 2003 .NET, but it's passing all its tests and the benchmarking I've been doing looks good, so I'm happy for the moment.

One problem I had initially was that the xsubpp output would not compile, complaining that cstdio.h and several other standard headers have syntax errors in them. That's odd, since these are standard headers distributed with the compiler -- it seems highly unlikely that they contain syntax errors, even if this is a Microsoft product.

I finally fixed it by putting the C++ header #includes first in the file and the standard Perl C header #includes after them. (and by making sure that the XS output was recognized as C++ rather than C, but that's tangential to my question). Other monks have run into very similar problems (here and here), and at least one person independently found the same solution.

Great. Problem solved. But why? Why should there be this restriction on the order of #included headers in XS modules? I can keep on using this solution in future XS extensions, but I'll feel like a member of some cargo cult if I keep doing it without understanding the why. So does any monk out there have a clue what's really going on here that the order of #includes has this effect?

--DrWhy

"If God had meant for us to think for ourselves he would have given us brains. Oh, wait..."

  • Comment on Why does order of #include's matter in XS file for C++ extensions under Windows?

Replies are listed 'Best First'.
Re: Why does order of #include's matter in XS file for C++ extensions under Windows?
by GrandFather (Saint) on Jul 24, 2006 at 19:31 UTC

    I don't know the specific headers involved, but the generall problem is one of dependencies. Headers are used to tell the compiler about stuff it needs to know so it can use library code that is not part of the current compilation unit. (Note my use of "library" is pretty loose here - I just mean stuff compiled outside the current compilation unit.)

    When different librarys are being used you can run into naming issues where different headers use the same identifier for different purposes. Often there is trickery in the headers to avoid some of the problems caused by that and often that trickery fails if headers are not included in the right order. This is most likely the issue that you were experiencing.

    Often headers for one compilation unit require other headers in order to compile. The header writer has a number of options:

    1. Include the required header
    2. Include the required header only if it hasn't been included already
    3. Assume the required header has already been included
    4. Define the stuff it needs so the other header is not required

    Depending on how the other header has been written, and how stable it is, all of those options can cause trouble! Probably the worst is 3 and possibly the best option is 2 closely followed by option 1.

    Writing headers can be pretty tricky stuff. Occasionally you get into a situation where two headers seem to depend on each other for example. Generally that is more likely to be a problem relating to C++ classes than to C headers and is generally related to classes that need to reference each other. Very often in that case you are seeing the result of poor design.

    Many headers still use the C preprocessor macro facility to provid named types. There is no way to manage such names and collisions between them to avoid "issues". In fact compile failures are much the best outcome when macros start wreaking havoc. It is much worse to have code that looks fine, but is behaving completely differently than you expect because some stupid macro has altered the meaning of an inocuous line of code into something that any obfusicated Perl hacker would be proud of.

    In the case of using other people's headers maybe cargo cult is best. :)


    DWIM is Perl's answer to Gödel
Re: Why does order of #include's matter in XS file for C++ extensions under Windows?
by whio (Beadle) on Jul 24, 2006 at 19:13 UTC
    When writing C or C++ code, there is a reasonable expectation that you are familiar with the language. One of the things that you need to do is always include system headers before program specific headers, because you may end up #defining tokens that are part of the system header files.

    My guess is that you are encountering conflicts between PerlIO (which optionally imports the Perl_ functions with the stdio function names), and #include <cstdio>, which imports the C standard I/O library into C++. The perl headers are all written for C, not C++. In fact, perlio.h has #include <stdio.h> inside it. It's perfectly reasonable to expect this to interact poorly with C++ code.

Re: Why does order of #include's matter in XS file for C++ extensions under Windows?
by philcrow (Priest) on Jul 24, 2006 at 18:57 UTC
    <disclaimer>I don't do windows and what follows is partly conjecture based on my understanding of C/C++.</disclaimer>

    In C, header files are always processed in the order the compiler finds them in. If a header of yours needs something from the standard headers and they have not been loaded yet, your header might define the thing it wants in a way it thinks will work. Having done that, the standard header may choose not to define the thing, since it sees that someone else has already done it. If the thing in question was a macro, syntax errors are possible.

    So, it is usually best to put any headers that came with the compiler first, then include others (like the ones that came with perl or the ones you wrote yourself).

    Phil