CGI::Application not disconnecting from the database?

jgallagher has asked for the wisdom of the Perl Monks concerning the following question:

I'm getting some really strange errors. I have a CGI::Application subclass that is a superclass for other, more specific CGI::App's. After a large number of tests (page reloads, etc.), I'll get an interal service error with the following log:

DBI connect('database','username',...) failed: Too many connections
[download]

Here is the relavent code in the class:

sub cgiapp_init {
    my $self = shift;

    # ...some unrelated code...

    my $dbh = DBI->connect('dbi:mysql:database', 'user', 'password',
                            { RaiseError => 1 } ) or die $DBI::errstr;

    # ...more code...

    $self->param('dbh' => $dbh);
}

sub teardown {
    my $self = shift;
    $self->param('dbh')->disconnect() or die $self->param('dbh')->errs
+tr;
}
[download]

The subclasses do not redefine cgiapp_init or teardown, and I can die in the teardown method, so I know it's getting to it. But apparently it's not disconnecting for some reason, unless somebody has any ideas on where else the problem might be. I've tried this with and without Apache::DBI.

Comment on CGI::Application not disconnecting from the database? Select or Download Code

Replies are listed 'Best First'.
Re: CGI::Application not disconnecting from the database? by lestrrat (Deacon) on May 08, 2003 at 08:04 UTC
Since you indicate that you tried with Apache::DBI, I'm guessing you're running this on mod_perl. If so, Apache might just be spawning a LOT of clients to deal with your requests. I think the easiest solution is to place a frontend proxy-only Apache, hide your mod_perl Apache from users, and set MaxClients to some low value that your database can withstand safely Of course, I don't know the entire picture, so I may be way off...	[reply]
Re: CGI::Application not disconnecting from the database? by CountZero (Bishop) on May 08, 2003 at 15:16 UTC
As you know the disconnect-method does not work with Apache::DBI. From the docs: There is no need to remove the disconnect statements from your code. They won't do anything because the Apache::DBI module overloads the disconnect method. Also the docs of Apache::DBI warn against opening too many connections which can easily happen: The Apache::DBI module still has a limitation: it keeps database connections persistent on a per process basis. The problem is, if a user accesses several times a database, the http requests will be handled very likely by different servers. Every server needs to do its own connect. It would be nice, if all servers could share the database handles. Currently this is not possible, because of the distinct name-space of every process. Also it is not possible to create a database handle upon startup of the httpd and then inheriting this handle to every subsequent server. This will cause clashes when the handle is used by two processes at the same time. With this limitation in mind, there are scenarios, where the usage of Apache::DBI is depreciated. Think about a heavy loaded Web-site where every user connects to the database with a unique userid. Every server would create many database handles each of which spawning a new backend process. In a short time this would kill the web server. The strange thing is that it happens also without using Apache::DBI. Are you sure that Apache::DBI is not started "silently" in the background by mod_perl's configuration? CountZero "If you have four groups working on a compiler, you'll get a 4-pass compiler." - Conway's Law	[reply]
Re: Re: CGI::Application not disconnecting from the database? by jgallagher (Pilgrim) on May 08, 2003 at 16:12 UTC
I printed out all of %INC, and there was no Apache::DBI to be found (I am using Apache::Session, but the only other DBI module was DBI.pm itself). I added the following before and after the disconnect call: `$self->param('dbh')->ping()` [download] It returned 1 both before and after. :( Any ideas?	[reply] [d/l]
Re: Re: Re: CGI::Application not disconnecting from the database? by CountZero (Bishop) on May 08, 2003 at 21:03 UTC
It sure seems that the disconnect-method is ... disconnected. On some Apache servers, the mod_perl configuration either contains a direct reference to Apache::DBI in the httpd.conf file or a reference to (e.g.) startup.pl which contains a `require Apache::DBI` statement. I'm not sure if Apache::DBI is `require`'d in that way, that it shows up in `%INC`. CountZero "If you have four groups working on a compiler, you'll get a 4-pass compiler." - Conway's Law	[reply] [d/l] [select]
Re: CGI::Application not disconnecting from the database? by Anonymous Monk on May 08, 2003 at 19:42 UTC
If you've done the checking, then it has nothing to do with CGI::Application specifically. In that case, it's purely a managment issue (your server doesn't allow you to have that many connections -- either allow more, or limit apache to fewer child processes, thus limiting the # of open DB connections).	[reply]