in reply to Do XS-components require special considerations with CGI? [SOLVED]

After what was (to me...) an extraordinary, week-long fight, I now conclude that this is, in fact, a memory-size problem. But finding the problem was a rather gruesome experience, as I would now like to briefly relate for the edification of anyone who may follow. I shall try to relate both epiphanies and mistakes ... and not to bore you.

Stave One:   the initial symptom


The initial symptom, Undefined subroutine &DBD::mysql::db::_login... actually occurs because a Perl Extension (XS) library routine failed to be loaded. As you can learn from perldoc DBI::DBD if you really enjoy looking at the “guts” of things, this module is supposed to be loaded by a call to DynaLoader::bootstrap at the start of DBD::mysql.pm. But it appears that this call was failing... and that any die-type messages that might have occurred are magically being “eaten.”

So I went right in to my local CPAN-module repository for this web-site, found the module, and wrapped the bootstrap call to become:

eval { ... bootstrap ... }; print STDERR "bootstrap result: $@\n";
This yielded the “unable to load” messages aforementioned, along with the first real clue (at least to poor me...) that it might somehow be memory-related.

A serious problem in a shared-hosting environment is that you don't get to see the Apache error logs. (And, as I discovered, any tools that they might provide to help diagnose server-errors might not actually provide the same environment.) I was able to force the STDERR output to go to a place I could actually use by employing use CGI::Carp qw(carpout). With this routine, in a BEGIN{} block, I could actually see the messages.

In order to actually discover what DynaLoader was doing ... I did this:

 $ENV{PERL_DL_DEBUG} = 1;

Stave Two:   red herrings...
Little fishes abound in any debugging chore, and this was especially true here. Nearly all of the helpful Internet postings concerning _login say that “the driver module cannot be found.” But I was able to eventually prove that the mysql.so module was being found, and that it was the right one. I spent a lot of time mucking-about with the LD_LIBRARY_PATH environment variable, chasing my way down this blind alley.

The program itself threw a herring at me because it didn't “flop over and die.” Instead, it caught the error and kept going. And yet, it might be said that it didn't catch the right error, because the bootstrap failure did not immediately reveal itself... only the consequences thereof.

Stave three:   zeroing-in on memory


As my (un-edited...) comments in these various threads will affirm, I didn't want to believe it really was memory, and I didn't give enough credence to the early admonition to use strace to discover it. For one thing, I didn't actually know what the resource limits were that applied to my code when it ran in CGI. This is the magic-spell that gave me that information:
print STDERR "Ulimit is:\n\t" . `bash -c "ulimit -a"` . "\nnow you know...\n";

The output (also listed earlier...) shows that the basic limit is 32meg, which I erroneously supposed must be “plenty.” Although I stumbled-upon the ulimit command (described in man bash ...) fairly early-on, I wasn't using it correctly.

There are two resource-limits that you can impose... “soft” and “hard.” But only the “soft” limit matters. So I wound up doing this:

bash # ... to give myself a nested shell ulimit -Sl32 # ... for example ulimit -a # ... to see what else I needed to change strace -ologfile perl spe-cgi.pl # to run the program with tracing # now review the "logfile" file that has magically appeared

When the program was sufficiently constrained, it finally began to fail, although it still does not do so in exactly the same way. (Right now it falls-dead.)

Replies are listed 'Best First'.
Re^2: Do XS-components require special considerations with CGI? [SOLVED]
by Anonymous Monk on Mar 04, 2009 at 07:48 UTC
    I'll bet if you examined %::, it would have shown mysql.so was not being loaded.

      Obviously I don't know about this variable or its uses. In a separate thread, perhaps, could you describe it?

        Obviously I don't know about this [%::] variable

        Quoting perlmod:

        The symbol table for a package happens to be stored in the hash of that name with two colons appended. The main symbol table's name is thus %main:: , or %:: for short.