The joy of socket communication

We all know that perl excels in all forms of communications ranging from STDIN/STDOUT/STDERR to POE. These include all methods supported by Unix, which means terminal IO, disk IO, pipes, message queues, shared memory, and lo and behold sockets.

In my working environments sockets are now used everywhere ranging from webservices to monitor programs and database communications.

Only two weeks ago, I tried to install DBD::Pg on a perl-5.10.1 tree with a freshly built postgres-8.4.5 whose test suite passed all tests so the odds looked like it would be a big success.

But DBD::Pg refused to connect to any postgres database server, neither local, nor remote. Numerous hours of debugging later, I found out that there are two kinds of sockets available on that system:

    Default DSB sockets:

      int getsockopt (
          int         s,
          int         level,
          int         optname,
          void       *optval,
          int        *optlen
          );

    UNIX 03 Only (X/Open Sockets):

      int getsockopt (
          int                    s,
          int                    level,
          int                    optname,
          void       *__restrict optval,
          socklen_t  *__restrict optlen
          );
[download]

The difference isn't that big, and you'll notice no difference in a 32bit environment, but in a 64bit environment, socklen_t is 64bit. You'll probably see the problems shine already. Perl itself was built with all default options, resulting in the standard BSD type sockets, whereas libpq from postgres was built with the X/Open type causing all socket calls that involve optlen to fail.

To prove that I was right, I compiled perl-5.12.2 from scratch, forcing it to use the X/Open type of socket calls. That involves one extra define and the addition of xnet to the libswanted during Configure. Perl's internal test suite passed all tests, and all my default modules (186 in total) built and installed fine.

Then came the final test: DBD::Pg, and all went well. Problem solved?

Of course not. Otherwise I wouldn't have posted here. We do not only use postgres databases, but we also use perl to connect to Unify, SQLite and CSV databases (I also test on mysql, but as we don't use it, I don't really care).

DBD::SQLite and DBD::CSV already passed in my basic setup, so that was fine. Next in line was the horrible DBD::Oracle. And that one showed the exact error that DBD::Pg showed when I started this quest: all connects failed.

If it were just for me, I wouldn't care. I hate Oracle. Not that there is a perfect database. In the end all databases suck, but Oracle and mysql suck more than the rest. But we have customers that use Oracle. A lot.

The problem now is that we also have scripts that connect to both Oracle and Postgres. That is now impossible. By now all of us probably know that you cannot use 32bit shared libraries in a 64bit environment and vice versa, but this is a completely new type of fail area.

If I have a BSD-socket type of perl, my Oracle communications work fine. I can make the Postgres scritps work by compiling DBD::Pg with the X/Open socket options and libraries and then setting export LD_PRELOAD=/usr/lib64/libxnet.so or wherever that library is located, but I cannot set that in my default environment, as the Oracle connections would fail. Neither is it possible to have script that communicate with both databases.

The life of IT people is so hard ...

(FWIW I currently have both perls installed at the customer site, and the script specifically sets the perl required. This does mean that I have to maintain both trees).

Enjoy, Have FUN! H.Merijn

Comment on The joy of socket communication Select or Download Code

Replies are listed 'Best First'.
Re: The joy of socket communication by mr_mischief (Monsignor) on Dec 20, 2010 at 10:46 UTC
First, let me point out that this has nothing to do with the joys of socket communications. It is sockets, in fact, that could bypass this sort of mess. This is the "joy" of linking to libraries from other projects in order to speak to them rather than using sockets. Both BSD libraries for sockets and X/Open libraries for sockets will intercommunicate over the actual sockets. You're just having problems trying to have both libraries linked into perl. If you could talk to the database servers using sockets directly rather than linking in their client libraries which use different socket libraries, then you'd only need either of the socket libraries linked into perl. Luckily, one of (Oracle\|Postgres) is itself Open Source. Would it be easier to specify that Postgres be made to accommodate the BSD sockets in Perl than to make Perl accommodate both Oracle and Postgres? How hard would it be to rebuild Postgres (or at least libpq)? Almost anything can be solved adding another layer of indirection. Yes, I'm about to suggest a dirty hack. Actually, more than one. Those who have eaten recently may want to look away. You could have your code check which database driver is going to be used. Then, if it's Postgres, it could alter the environment variables for the libraries to preload. Then, it could `exec` the actual process that will connect to the database. You could do this by launching it from a shell script with the export, or by actually setting the proper values in %ENV in Perl and then exec()ing. This alone doesn't get you talking to both databases, but it does get you loading the incompatible X/Open socket library only in a branch of your process tree. In yet another layer of indirection, you could have one process doctored to speak to Postgres as above and one left alone (since your master environment isn't bothered) to talk to Oracle. Then, you could have a third process which coordinates with both of the other two through sockets instead of through library linkages. Yes, this makes your application two processes larger than it was and more complex. You'd need to figure out a protocol for moving data back and forth across the sockets. There are such projects out in the wild as DBD::Proxy/DBD::ProxyServer and DBD::Gofer, but they seem to come with their own issues. After all of the last two paragraphs, is it still sounding crazy to rebuild libpq linked against the BSD socket libraries? Are the Postgres and perl in question both supplied by your OS vendor? It seems very odd that one vendor would choose the X/Open libraries in one case and BSD libraries in the other in the same system, especially for things commonly linked together like perl and Postgres client libs. Perhaps you should report an integration bug. Do the `ldd` outputs for libpq and for Pg.so built with an unadulterated perl vary wildly in what libraries they list? This is, in fact, not a new failure area at all. Incompatible structure alignments and function definitions due to argument lengths from different implementations of a standard are par for the course in C development. The secret is to not need to use two incompatible versions of the same function from two similar but different libraries in the same program.	[reply] [d/l] [select]
Re^2: The joy of socket communication by Tux (Canon) on Dec 20, 2010 at 11:23 UTC
I've just finished (yeah, just 20 minutes ago) a complete test in which I managed to force-build postgres in BSD type socket interface, passing both its own test suite and DBD::Pg's test suite. In another window, I am writing a mail to the postgres developers explaining what I did, why is is hard to do so, and some suggestions in how they could ease up the build chain to accommodate these changes. The basic (postgres) problem seems to lie in a fix that was explicitly added to make socket communication work on PA-RISC, which is not valid (anymore) on IA64. Enjoy, Have FUN! H.Merijn	[reply]
Re: The joy of socket communication by BrowserUk (Patriarch) on Dec 20, 2010 at 11:36 UTC
I've no way to test this, nor enough knowledge of the relevant systems to even reason about it, but before I could get DBD::Pg to build on my win64 setup, I successfully used DBD::PgPP. Maybe that would work with your oracle capable build? Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice.	[reply]