Author RE: ANNOUNCE: IBM Informix Database Driver for Perl DBI Version 2
Brian D Campbell
2005-08-11, 5:02 pm
My huge reply to this thread (which is a problem/solution report) appears below the ==== separator line. If you found this email posting to be
Helpful, please drop me an email.
- Brian
________________________________________
From: Jonathan Leffler mailto:jonathan.leffler@gmail.com
Sent: Wednesday, August 10, 2005 4:53 PM
To: CAMPBELL, BRIAN D (BRIAN)
Cc: DBI Users Mailing List
Subject: Re: Informix DBD build on Windows - Better
-----Original Message-----
On 8/8/05, CAMPBELL, BRIAN D (BRIAN) <campbelb@lucent.com> wrote:
All in all I uncovered 9 bugs in the build process for DBD.
Should I go ahead and publish my findings to this mailing list (DBI
users)?
As Ron said ("Publish and Be Damned")
Yes - please post bug reports - preferably with the patches to fix them - to the list. They may help other people; eventually, they'll help me help you.
As discussed wholly separately, t08fork.t failures indicate problems in the test, and t91udts.t failures indicate that the database wasn't as clean as it should be (arguably a test problem in itself). Not dreadfully significant, though both should idea
lly be fixed.
-----Original Message-----
From: CAMPBELL, BRIAN D (BRIAN) mailto:campbelb@lucent.com
Sent: Wednesday, August 03, 2005 4:42 PM
To: 'dbi-users@perl.org'
I tried building the recently released Informix DBD 2005.02 today. I apparently have a successful compile and link. But I'm having a major problem on the test. Below is the link input and output, and the first test. All remaining tests look like the fi
rst test. Help and suggestions needed. Here's the message I'm getting upon running the first test (and all the rest).
Can't find 'boot_DBD__Informix' symbol in ./Informix.dll
stuff snipped
My Environment:
* Windows 2000 Version 5.00.2195 SP4
* ActiveState Perl v5.8.7 built for MSWin32-x86-multi-thread, Binary build 813 148120
* DBI: 1.48
* ESQL: IBM Informix CSDK Version 2.90, IBM Informix-ESQL Version 2.90.TC1
* Microsoft Visual C++ 6.0
* Connection to Informix DB server 9.21.FC4 on an Alpha box running Compaq Tru64 Unix.
========================================
================================
This posting documents the problems I had while trying to build the recently
released Informix DBD 2005.02 on Windows.
First of all, the makefile generator produces this notice for Windows systems:
This is a semi-experimental version of Makefile.PL with Win32 (NT)
support. Previous versions have been tested on NT, but there could
still be bugs in this code. Please help save the world by trying
to debug the NT code and reporting the results back to the DBI
Users mailing list.
So once again, here I go, saving the world...
In the documetation below, I use the following "pattern language" for
each problem:
The symptom - the message from the console.
The reason - my best guess on what is causing the symptom.
A workaround (sometimes) - I provided an expediant "Workaround", which
I used to keep going.
A recommended solution - Usually just a description and nothing as fancy
as a real patch.
A preliminary remark: I recently upgraded my Perl to 5.8.7. In doing so,
I took my own previous advice and installed it in "C:\Perl" which I believe
is the default path provided by the ActiveState installer. Previously,
it was installed in "C:\Program Files\Perl", per a local site installation
process. Having that blank char in the Perl path caused all kinds of grief
in previous build attempts, as well documented in my previous post on the
subject:
http://www.mail-archive.com/dbi-use...g/msg19771.html
The majority of problems I encountered then had that one root cause.
The build process uses several Perl modules to due some preliminary set
up; these modules were not good at handling spaces in file paths.
Thus, I was able to avoid a class of problems I had encountered previously.
It was also helpful to use my Batch file tip and CPAN/Config.pm tip as
described near the bottom of that posting.
OK, now on to the problem set...
Problem 1
=========
Symptom:
When running the makefile generator (Makefile.pl):
This warning occurs:
Note: Some of the code here or in ESQL/C may work properly if
%INFORMIXDIR% contains spaces.
Then I got this:
'C:\Program' is not recognized as an internal or external command, operable program or batch file.
Reason:
The cautionary note is true about the "spaces" concern, but is mistaken
about the env var when running under Windows. It didn't help when I changed
my INFORMIXDIR var from:
C:\Program Files\Informix\Client-SDK
to:
C:\Progra~1\Informix\Client-SDK
The script's path of execution for Windows does not look at the %INFORMIXDIR%
env var. Instead, the script locates the Informix folder value by searching
all folders in the PATH list, testing if ESQL.EXE is located in any of them,
and then trims \BIN from the discovery.
The problem with this "PATH search" approach is that the PATH is typically
set by the Informix SDK installation using the file name with blanks.
Workaround:
One can update their PATH setting to remove the blanks or do what I did.
I put the perl Makefile.pl in a batch script, and temporarily altered the
PATH list to include my preferred path as follows. It used my path instead
of the installation path, because mine comes first.
SET INFORMIXDIR=C:\Progra~1\Informix\Client-SDK
PATH %INFORMIXDIR%\bin;%PATH%
perl Makefile.pl
Recommended solution:
When running under Windows:
The Makefile.pl script should consider using INFORMIXDIR, if present.
The script should at least test for blanks and complain when blanks are found.
Or better yet, translate the path to the alternate 8.3 "blankless" name
(Win32::GetShortPathName may be of help here).
Problem 2
=========
Sympton:
When running the makefile:
dbdimp.ec(32) : fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory
Reason:
File dbdimp.ec has this, which doesn't compile in MSVC.
#include <unistd.h>
Recommended solution:
Replace the line with the following solution found in ixblob.ec.
#ifdef _WIN32
#include <io.h>
#else
#include <unistd.h>
#endif /* _WIN32 */
Problem 3
=========
Symptom:
When running the makefile:
ixblob.ec(126) : error C2065: 'F_OK' : undeclared identifier
Reason:
File ixblob.ec refers to the undefined symbol F_OK, whis is passed to the
function access (declared in io.h). My verions of MSVC doesn't define the
symbol.
Recommended solution:
Add this to the file. The definition comes from my Unix platform in
<sys/access.h>. It would seem that the same defined value works on Windows
too.
#ifdef _WIN32
#define F_OK 00 /* E_ACC does file exist */
#endif /* _WIN32 */
Problem 4
=========
Symptom:
When running the makefile:
'INFORMIXC' is not recognized as an internal or external command, operable program or batch file.
Reason:
This line is in the makefile.
LD = INFORMIXC='$(FULLPERL) esqlld' ESQLLD='link $(LDDLFLAGS)' $(LDCMD)
This defines a system command which includes the INFORMIXC and ESQLLD as
Unix-style shell parameters preceding the command $(LDCMD). A reference to
LD is here in the Informix.dll target $(INST_DYNAMIC). Shell parameters
don't work on Windows.
$(INST_DYNAMIC): $(OBJECT) $(MYEXTLIB) $(BOOTSTRAP) $(INST_ARCHAUTODIR)$(DIRFILESEP).exists $(EXPORT_LIST) $(PERL_ARCHIVE) $(INST_DYNAMIC_DEP)
$(LD) -out:$@ $(LDDLFLAGS) $(LDFROM) $(OTHERLDFLAGS) $(MYEXTLIB) $(PERL_ARCHIVE) $(LDLOADLIBS) -def:$(EXPORT_LIST)
$(CHMOD) $(PERM_RWX) $@
Workaround:
Remove the paramaters from the LD definition:
LD = $(LDCMD)
Modify the Informix DLL target (where LD is referenced) to have ESQLLD and ESQLLD
on separate lines (in DOS style without quotes).
$(INST_DYNAMIC): $(OBJECT) $(MYEXTLIB) $(BOOTSTRAP) $(INST_ARCHAUTODIR)$(DIRFILESEP).exists $(EXPORT_LIST) $(PERL_ARCHIVE) $(INST_DYNAMIC_DEP)
SET INFORMIXC=$(FULLPERL) esqlld
SET ESQLLD=link $(LDDLFLAGS)
$(LD) -out:$@ $(LDDLFLAGS) $(LDFROM) $(OTHERLDFLAGS) $(MYEXTLIB) $(PERL_ARCHIVE) $(LDLOADLIBS) -def:$(EXPORT_LIST)
$(CHMOD) $(PERM_RWX) $@
Recommended solution:
For Windows systems, have the make file generator Makefile.pl create the
makefile lines as described above.
Problem 5
=========
Symptom:
When running the makefile:
'-out' is not recognized as an internal or external command, operable program or batch file.
Or at least I think that's what I got (I don't have a saved record of this
this portion of my build).
Reason:
Makefile sets the LD command using this:
LD = INFORMIXC='$(FULLPERL) esqlld' ESQLLD='link $(LDDLFLAGS)' $(LDCMD)
Or modified as this, using the solution from Problem 4:
LD = $(LDCMD)
And later in the file, this sets LDCMD to the ESQLC "wrapper" around the linker.
LDCMD = $(ESQL)
The linker command LDCMD is defined late in the file, so variable LD sees
the ESQLC command value as an undefined empty string.
Workaround:
Move the LDCMD= definition before the LD= definition.
Or just set LD to $(ESQL).
Or just replace $(LD) with $(ESQL) in the Informix.dll target.
Recommended solution:
Have the make file generator Makefile.pl generate a makefile
using one of the above alternatives.
Problem 6
=========
Symptom:
When running the makefile:
msvcrt.lib(crtexe.obj) : error LNK2001: unresolved external symbol _main
Informix.exe : fatal error LNK1120: 1 unresolved externals
In the makefile target, described above, LDDLFLAGS includes the option:
-dll
Which is the incorrect ESQLC option for creating .DLL. Thus, it tries to create
a complete program and complains because an implementation for "main()" can't
be found.
Workaround:
Change the option in LDDLFLAGS to be:
-target:dll
As documented in IBM publication:
Informix-ESQL/C, Programmer's Supplement for Microsoft Windows Environments
Recommended solution:
Have the make file generator Makefile.pl define LDDLFLAGS as described above.
Problem 7
=========
Symptom:
When running the supplied tests, virtually every test file returned something
like this:
t/t00basic..........install_driver(Informix) failed: Can't find 'boot_DBD__Informix' symbol in ./Informix.dll
at (eval 1) line 3
Compilation failed in require at (eval 1) line 3.
at D:\build\DBD\DBD-Informix-2005.02\blib\lib/DBD/Informix/TestHarness.pm line 185
t/t00basic..........dubious
Test returned status 2 (wstat 512, 0x200)
DIED. FAILED tests 1-41
Failed 41/41 tests, 0.00% okay
Reason:
It took me a while to dig into this one. I had to understand, to some extent,
what was going in on the code, in the file Informix.c. This is file is
automatically generated from a "XS" language in Informix.xs and uses macros
extensively. At the suggestion of a colleague, I ran the c-preprocessor to see
what was really being compiled in the Informix.c file. And what symbols were
put in the .DLL by looking at the link map. Long story short, all the symbols
in Informix.c were omitted from the DLL because a required "Informix.def" file
was NOT properly passed to the MSVC linker. That was because it was not
properly specifed on the ESQL command.
These lines occur in the makefile:
BASEEXT = Informix
EXPORT_LIST = $(BASEEXT).def
$(LD) -out:$@ $(LDDLFLAGS) $(LDFROM) $(OTHERLDFLAGS) $(MYEXTLIB) $(PERL_ARCHIVE) $(LDLOADLIBS) -def:$(EXPORT_LIST)
The token -def: should not appear. The argument is just supposed to be a bare
file according to the IBM publication:
Informix-ESQL/C, Programmer's Supplement for Microsoft Windows Environments
Workaround:
I expect that just removing the -def: prefix from the EXPORT_LIST reference,
above, should work. But before I spotted that in the file, my solution was
to add the file to the LDDLFLAGS variable:
LDDLFLAGS = ...stuff here... Informix.def
Recommended solution:
Have the make file generator Makefile.pl define the def option as described
above (remove the prefix -def:).
Problem 8
=========
Symptom:
When running the supplied tests, virtually every test file failed. The Informix
DLL was crashing. I don't have an exact record of this portion of the build,
but the symptom look similar to that of Problem 7.
Background and Reason:
I recognized this problem, having patched it in previous builds of the
Informix DBD I have built in 2003 and 2004. I think it's crashing because
it's freeing memory from a different pool (using free) that it was allocated
(needs to use SqlFreeMem instead per PTS Bug B83831).
Workaround:
In dbdimp.ec add this above the function: count_descriptors
#define PERL_OBJECT
This must be place "low" in the file; placing this "high" in the file cause
compile errors on includes.
Recommended Solution:
Not sure. Add this to the dbdimp.ec file? (But not too high in the file.)
#ifdef _WIN32
#define PERL_OBJECT
#endif /* _WIN32 */
Problem 9
=========
Symptom:
DBI Can't load Informix DBD in my own scripts.
Reason:
When running the installation process, 3 files did not get copied to:
C:\Perl\site\libs\auto\DBD
I think they needed to be in a similar path in the build folder I was using.
I don't think the makefile put them there; it left them in the top level of
the build folder.
Workaround:
I copied them manually. They were.
Informix.dll
Informix.exp
Informix.lib
Recommended Solution:
The makefile needs to ensure the link output gets into the proper place in the
build folder before the installation step.
Other Issues
============
All tests passed, except 4 were skipped and two tests had failures:
t\t08fork.t
t\t91udts.t
The first test produced this message:
Informix has no driver CLONE() function so is unsafe threaded DBD::Informix::db prepare failed: handle 2 is owned by thread 15e6014 not current thread 214b68c (handles can't be shared between threads and your driver may need a CLONE method added) at t\t08
fork.t line 32.
This message is cryptic to me, but suggests to me that an Informix connection
can't span multiple threads.
Several messages from the 2nd test appeared similar to:
DBD::Informix::db do failed: SQL: -9991: Named row type (dbd_ix_udts_named) already exists in database. at D:\build\DBD\DBD-Informix-2005.02\blib\lib/DBD/Informix/TestHarness.pm line 347.
This and other output suggests to me that the testplan can't clean up User
defined data types in the database that were created in an earlier test.
It wants to clean them up, in order to recreate them in a new test.
That's it.