syphilis has asked for the wisdom of the Perl Monks concerning the following question:

Hi,

This is actually an issue wrt my 64-bit PDL binaries when they get installed on Strawberry Perl.
The distributed auto/PDL/Minuit/Minuit.dll and auto/PDL/Slatec/Slatec.dll need to load the libgcc_s_sjlj-1.dll from the gcc-4.7.0 compiler that I used to build both perl and PDL - and a copy of that libgcc_s_sjlj-1.dll therefore sits alongside those 2 dll's in the distributed package. Instead, however, they use Strawberry's libgcc_s_sjlj-1.dll (which was already loaded by Strawberry Perl itself and is from the gcc-4.4.7 compiler), resulting in an "unfound entry point" error.

So ... is there some trick I can use to force Minuit.dll and Slatec.dll to load the libgcc_s_sjlj-1.dll that sits next to them, instead of using the libgcc_s_sjlj-1.dll that Strawberry perl has already loaded ?
(Note that, on Windows, if libgcc_s_sjlj-1.dll had not already been loaded, then Minuit.dll and Slatec.dll would automatically load the desired libgcc_s_sjlj-1.dll, irrespective of any path settings.)

I wondered initially about the possibility of using Win32::FreeLibrary() to unload the Strawberry dll, followed by a Win32::LoadLibrary() call that specifically loads the libgcc_s_sjlj-1.dll that's needed.
But then, that would mean that Strawberry Perl itself would then be accessing the wrong libgcc_s_sjlj-1.dll, wouldn't it ?
And I wonder whether unloading such a crucial dll would be possible in the first place - even if I did manage to find the handle to it that Win32::FreeLibrary needs.

One simple solution is to replace the libgcc_s_sjlj-1.dll that Strawberry Perl loads (namely, its perl/bin/libgcc_s_sjlj-1.dll) with the libgcc_s_sjlj-1.dll that I'm distributing.
And that seems to work ... but at what risks ?

Another obvious solution is for me to use the same compiler as Strawberry Perl uses - but there's a fair degree of inconvenience associated with that solution, and I would rather not do it if an alternative exists. (In fact, I probably *won't* do it, even if it *is* the only proper solution :-)

Cheers,
Rob

Replies are listed 'Best First'.
Re: [OT][Win32] How to load the right dll
by BrowserUk (Patriarch) on Mar 20, 2012 at 02:41 UTC

    The MS solution to your problem is called side-by-side assemblies.

    In a nutshell, it is the "manifest file" mechanism that we all know (and hate:). You need to add the name of the dependant dll to the manifests embedded within the dlls dependant upon it.

    There is some (completely opaque) discussion about manifests here. And some reference material for the mt.exe tool here.

    Whether Mt.exe work with Mingw-built dlls? Probably, but I don't know for sure.

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

    The start of some sanity?
[reply]
      Thanks for the reply and the links.

      Whether Mt.exe work with Mingw-built dlls? Probably, but I don't know for sure

      I think it works ok - though so far all I've established by using it is that none of these dll's have resource sections.

      I don't know why I didn't realize it sooner, but not long after I posted here it occurred to me that I could frame my questions in such a way that it would not be OT on the mingw64 mailing list - so I posted there, too.
      The same approach has been suggested there.

      It was also suggested that replacing the problem libgcc_s_sjlj-1.dll with the one from the 4.7.0 compiler is probably not the disaster-just-waiting-to-happen that I was worried about ... so there's the fallback solution :-)

      Cheers,
      Rob
[reply]

        I made this work once a couple or so years ago, but I've forgotten something.

        Four files: main.exe link-time linked to a.dll; runtime links to dll\b.dll which is link-time linked to dll\a.dll. The a.dlls differ.

        The sources:

        #include <stdio.h>
#define  ISOLATION_AWARE_ENABLED 1
#include <windows.h>

__declspec(dllimport) int funca( int i );


int main( int argc, char **argv ) {
    char buf[ 100 ];
    HMODULE hb = LoadLibrary( "dll\\b.dll" );
    int (*b)(int) = GetProcAddress( hb, "funcb" );

    printf( "hmodule: %x *funcb:%x \n", hb, b );
    funca( 1 );

    printf( "Back in main after calling locally bound funca\n"); gets(
+ buf );
    b( 2 );

    return 0;
}

/* a.dll (v1)*/
#include <stdio.h>

__declspec(dllexport) int  _stdcall funca( int i ) {
    return printf( "a.dll{v1.0.0.0}:funca() called with %d\n", i );
}

/* b.dll */
#include <stdio.h>

__declspec(dllexport) int funcb( int i ) {
    printf( "b.dll:funcb() called with %d\n", i );
    funca( i + 1 );
    return 1;
}

/* a.dll (v2)*/
#include <stdio.h>

__declspec(dllexport) int funca( int i ) {
    return printf( "a.dll{v2.0.0.0}:funca() called with %d\n", i );
}

        [download]

        And these (handwritten) manifest files for the two a.dlls:

        <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.
+0">

<assemblyIdentity type="win32" name="a.dll" version="1.0.0.0" />
<description></description>
</assembly>

        [download]
        <?xml version="1.0" encoding="UTF-8" standalone="yes"?>

<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.
+0">

<assemblyIdentity type="win32" name="a.dll" version="2.0.0.0" />
<description></description>
</assembly>

        [download]

        The other two are generated by the linker; and are attached to their executables using mt.exe)

        And then build them all using this:

        @rem makeit.cmd
del /S /Q *.obj *.lib *.dll *.exe

cd dll
cl /MT /LD a.c 
mt -manifest a.dll.manifest -outputresource:a.dll;#1

cl /MT /LD b.c a.lib /link /manifest "/manifestdependency:type='win32'
+ name='a.dll' version='2.0.0.0'"
mt -manifest b.dll.manifest -outputresource:b.dll;#1

cd .. 
cl /MT /LD a.c 
mt -manifest a.dll.manifest -outputresource:a.dll;#1

cl /MT main.c a.lib /link /manifest "/manifestdependency:type='win32' 
+name='a.dll' version='1.0.0.0'"
mt -manifest main.exe.manifest -outputresource:main.exe;#1

        [download]

        like so:

        C:\test\manifest>makeit

C:\test\manifest>del /S /Q *.obj *.lib *.dll *.exe
Deleted file - C:\test\manifest\a.obj
Deleted file - C:\test\manifest\main.obj
Deleted file - C:\test\manifest\a.lib
Deleted file - C:\test\manifest\a.dll
Deleted file - C:\test\manifest\main.exe
Deleted file - C:\test\manifest\dll\a.obj
Deleted file - C:\test\manifest\dll\b.obj
Deleted file - C:\test\manifest\dll\a.lib
Deleted file - C:\test\manifest\dll\b.lib
Deleted file - C:\test\manifest\dll\a.dll
Deleted file - C:\test\manifest\dll\b.dll

C:\test\manifest>cd dll

C:\test\manifest\dll>cl /MT /LD a.c
Microsoft (R) C/C++ Optimizing Compiler Version 15.00.21022.08 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

a.c
Microsoft (R) Incremental Linker Version 9.00.21022.08
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:a.dll
/dll
/implib:a.lib
a.obj
   Creating library a.lib and object a.exp

C:\test\manifest\dll>mt -manifest a.dll.manifest -outputresource:a.dll
+;#1
Microsoft (R) Manifest Tool version 5.2.3790.2075
Copyright (c) Microsoft Corporation 2005.
All rights reserved.

C:\test\manifest\dll>cl /MT /LD b.c a.lib /link /manifest "/manifestde
+pendency:type='win32' name='a.dll' version='2.0.0.0'"
Microsoft (R) C/C++ Optimizing Compiler Version 15.00.21022.08 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

b.c
Microsoft (R) Incremental Linker Version 9.00.21022.08
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:b.dll
/dll
/implib:b.lib
/manifest
"/manifestdependency:type='win32' name='a.dll' version='2.0.0.0'"
b.obj
a.lib
   Creating library b.lib and object b.exp

C:\test\manifest\dll>mt -manifest b.dll.manifest -outputresource:b.dll
+;#1
Microsoft (R) Manifest Tool version 5.2.3790.2075
Copyright (c) Microsoft Corporation 2005.
All rights reserved.

C:\test\manifest\dll>cd ..

C:\test\manifest>cl /MT /LD a.c
Microsoft (R) C/C++ Optimizing Compiler Version 15.00.21022.08 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

a.c
Microsoft (R) Incremental Linker Version 9.00.21022.08
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:a.dll
/dll
/implib:a.lib
a.obj
   Creating library a.lib and object a.exp

C:\test\manifest>mt -manifest a.dll.manifest -outputresource:a.dll;#1
Microsoft (R) Manifest Tool version 5.2.3790.2075
Copyright (c) Microsoft Corporation 2005.
All rights reserved.

C:\test\manifest>cl /MT main.c a.lib /link /manifest "/manifestdepende
+ncy:type='win32' name='a.dll' version='1.0.0.0'"
Microsoft (R) C/C++ Optimizing Compiler Version 15.00.21022.08 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

main.c
main.c(11) : warning C4113: 'FARPROC' differs in parameter lists from 
+'int (__cdecl *)(int)'
main.c(11) : warning C4133: 'initializing' : incompatible types - from
+ 'FARPROC' to 'int (__cdecl *)(int)'
Microsoft (R) Incremental Linker Version 9.00.21022.08
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:main.exe
/manifest
"/manifestdependency:type='win32' name='a.dll' version='1.0.0.0'"
main.obj
a.lib

C:\test\manifest>mt -manifest main.exe.manifest -outputresource:main.e
+xe;#1
Microsoft (R) Manifest Tool version 5.2.3790.2075
Copyright (c) Microsoft Corporation 2005.
All rights reserved.

        [download]

        The run the exe and it 'works' in very way ... except the crucial one of dynamically lining to the right a.dll from b.dll:

        C:\test\manifest>main
hmodule: 640000 *funcb:641000
a.dll{v1.0.0.0}:funca() called with 1
Back in main after calling locally bound funca

b.dll:funcb() called with 2
a.dll{v1.0.0.0}:funca() called with 3

        [download]

        Which is a crap load of work to achieve the same result as you'd get without the manifest files, but I do remember when I first tried this I had to read gobloads of information and tear my hair out for several days to reach this point. So this might help you get started...

        The solution is something to do with "activation contexts", but I'm to tired to wrap my brain around it right now. Maybe tomorrow.

        HTH

        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

        The start of some sanity?
[reply]
[d/l]
[select]
Re: [OT][Win32] How to load the right dll
by bulk88 (Priest) on Mar 20, 2012 at 05:09 UTC
    I dont know what PDL is and Im not googling it this moment. What is the function that your libgcc 4.7.0 linked dlls are trying to load that isn't in libgcc 4.4.7? Any reason you can't compile Minuit.dll and Slatec.dll with strawberry perl's gcc?

    need to load the libgcc_s_sjlj-1.dll from the gcc-4.7.0 compiler that I used to build both perl and PDL - and a copy of that libgcc_s_sjlj-1.dll therefore sits alongside those 2 dll's in the distributed package. Instead, however, they use Strawberry's libgcc_s_sjlj-1.dll (which was already loaded by Strawberry Perl itself and is from the gcc-4.4.7 compiler), resulting in an "unfound entry point" error.

    You said you built your own perl. I would assume then that was built with gcc 4.7.0, not strawberry perl's 4.4.* (???) gcc. You need to remove strawberry perl from your path. Your pushing the limits of, or breaking ABI compatibility. GetModuleHandle() or GetModuleHandleEx() the C call will return to you a HMODULE to your libgcc, not sure if it exists anywhere in XS off the top of my head. Now lets assume you are serious about swapping DLLs at runtime. If you FreeLibrary it, in theory, you will need to patch every other DLL's import table/PE image that links to libgcc with new libgcc function pointers. You WONT be able to patch anything that getproced libgcc. You'll probably have to hotpatch/redirect that old libgcc to the new one, quite doable, but your verging on virus/spyware programing :D Read about the PE format and RVAs. Plenty of C code required.

    Looking in my various mingw builds, I see a libgcc.a at 3 MB, it has assembly code in it, I see a libgcc_eh.a at 42KB, and a libgcc_s.a at 74 KB. Google says the -static-libgcc will remove your dependency on libgcc.

    You could also try to loadlibrary/getprocaddress in C/XS your problem away. LoadLibrary will load your specific DLL if you specify a full path and even if that DLL by file name has already been loaded. You probably will want to try to compile with -nostartfiles in gcc and see how what calls gcc wants from libgcc.dll. GNU indent post preprocessor C code is an excellent diagnostic tool when trying to nail down linking problems. I've never gotten SXS to work to any degree for me. It probably wont work for you because your libgccs arent manifested. Your dlls would have to sit in the winsxs folder. Maybe be in the registry. I've never gotten windows to honor a .manifest file in a folder. Use a PE image viewer and look at the import table of your dlls, and the export tables of various dlls.

    Another idea would be to "embed" strawberry perl, then loadlibrary strawberry perl's perl51X.dll. This way your 4.7.0 libgcc will be in memory before perl has a change to load. Since S Perl's libgcc is older, is probably will load fine with a newer libgcc than it was compiled with. You could just copy over S Perl's libgcc dll with your 4.7.0 and see what happens. Is this for yourself, your company/employer, or CPAN quality?

    Plenty of grammer and sentence mistakes above. I'm not reading twice what I wrote. I gave you alot of ideas. Sane and insane.
[reply]
      I dont know what PDL is

      Sorry, I should've linked.

      What is the function that your libgcc 4.7.0 linked dlls are trying to load that isn't in libgcc 4.4.7?

      It's __addtf3. (Presumably that's just the first one that's found ... if that were to be resolved we might then find there are others ?)

      Your pushing the limits of, or breaking ABI compatibility

      No ... I mean if gcc-4.4.7 and gcc-4.7.0 didn't both name the dll 'libgcc_s_sjlj-1.dll' there would be no problem.
      (Hmmm ... on further reflection I guess that doesn't disprove the notion that I'm "pushing the limits".)

      the -static-libgcc will remove your dependency on libgcc

      I wonder ... would I have to build perl with that flag for that to work ? Or could I just include that flag for the building of PDL, without first rebuilding perl ?
      In a few places I've seen recommendations against using that flag where perl is concerned ... but I don't know the reasoning.

      Is this for yourself, your company/employer, or CPAN quality?

      It's for anyone who wants it.

      Plenty of grammer and sentence mistakes above

      Didn't spot any mistakes, except for the spelling of 'grammar' ;-)

      I gave you alot of ideas. Sane and insane.

      For which I thank you !!

      Cheers,
      Rob
[reply]
        It's __addtf3. (Presumably that's just the first one that's found ... if that were to be resolved we might then find there are others ?) No ... I mean if gcc-4.4.7 and gcc-4.7.0 didn't both name the dll 'libgcc_s_sjlj-1.dll' there would be no problem. (Hmmm ... on further reflection I guess that doesn't disprove the notion that I'm "pushing the limits".)

        I couldn't find __addtf3 being used PDL's sources. per http://gcc.gnu.org/onlinedocs/gccint/Soft-float-library-routines.html its a backup for older CPUs.

        the -static-libgcc will remove your dependency on libgcc

        ___________________________
        I wonder ... would I have to build perl with that flag for that to work ? Or could I just include that flag for the building of PDL, without first rebuilding perl ? In a few places I've seen recommendations against using that flag where perl is concerned ... but I don't know the reasoning.

        Compile your XS PDL library with that flag to make libgcc static. Or use the compiler that came with the strawberry perl version your targeting since this is for public distribution. You haven't said that gcc 4.4.X specifically doesn't work for you and 4.7.x does. I would recompile with the compiler strawberry perl came with. Perl specifically doesn't support compatibility between major versions for XS libraries (5.12 vs 5.14, 5.10 vs 5.12, and so forth). The headers change. The struct layouts change. Last I heard, Strawberry Perl and MSVC Perl of the same perl version do have ABI compatibility. PDL seems to be C only, not C++, so libgcc's exception handling (*unwind* calls, *frame* calls) shouldnt be needed, just long byte wise math ops. Apparently, GCC allows you to supply your own math functions perl http://gcc.gnu.org/onlinedocs/gccint/Soft-float-library-routines.html so in theory a Loadlibrary should be possible, but I suggest using the compiler that came with the perl, or static libgcc. You could also use dlltool and generate a new import library which is identical to 4.7.0 libgcc with a different DLL name such as libgccPerlPDL.dll. The import library will be 32/64 specific. It is possible to generate one on the fly from a makefile or from makefile.pl. Use perl to capture and parse the output of "`nm -g C:\folder\libgccforsharedlibraries.a`" and use that output, after regex processing, to spit out a def file with the symbols you found. Feed the def file into dlltool, dlltool will generate a .a file. Include that .a file along with -nostartfiles (you dont want the compiler's libgcc.a to be found) to gcc linker phase.
[reply]
Re: [OT][Win32] How to load the right dll
by syphilis (Archbishop) on Mar 21, 2012 at 01:05 UTC
    So ... is there some trick I can use to force Minuit.dll and Slatec.dll to load the libgcc_s_sjlj-1.dll that sits next to them, instead of using the libgcc_s_sjlj-1.dll that Strawberry perl has already loaded ?

    The answer is "yes" - and it turns out to be a very simple trick, which comes courtesy of Mark Dootson (again):

    Firstly rename libgcc_s_sjlj-1.dll to something that has the same number of characters (eg libgcc_sis__470.dll).
    Secondly, in Slatec.dll and Minuit.dll 
    s/libgcc_s_sjlj\-1\.dll/libgcc_sis__470\.dll/g

    [download]
    That done, and as long as Slatec.dll and Minuit.dll can find libgcc_sis__470.dll, there's no problem.

    I don't know how that sort of fix sits with others - but it fits right in with the way I like to do things.

    Mark did say he had first tried solving the problem using manifests, but found that "that private manifests (load a dll from a specific location) only work at the whole application level" - which put an end to his endeavours with manifests.

    Thanks for the inputs from BrowserUk and bulk88.
    I feel bad when people put a lot of effort into helping me out with various approaches .... and then I go and use a completely different solution :-(

    Cheers,
    Rob
[reply]
[d/l]

      I feel bad when people put a lot of effort into helping me out with various approaches .... and then I go and use a completely different solution :-(

      Don't feel bad, your username was fair warning :)

[reply]

Back to Seekers of Perl Wisdom