Re^2: How to debug "Too Many Open Files" Error

Thanks for the very thorough answer. Here is a typical extract from my error log:

[Tue Sep 16 06:47:33 2008] [error] [client 65.5.128.20] (24)Too many o
+pen files: couldn't spawn child process: /var/www/vhosts/cfhs69.com/c
+gi-bin/alumni001A.pl
[download]

Based on your comment, this would imply that there are too many open files in my process, not in the system. The difficulty for me is that I do not open files directly. I have one OPEN command that is for MAIL, and this module has not been involved in the failures. I use some other PERL modules that may open files, such as Authen::Captcha, but I assume they are well behaved. And I only use this once per user. And I only have 88 users and I would be surprised if there is ever more than 2 using the site at once. Tony

Comment on Re^2: How to debug "Too Many Open Files" Error Download Code

Replies are listed 'Best First'.
Re^3: How to debug "Too Many Open Files" Error by tilly (Archbishop) on Sep 17, 2008 at 03:56 UTC
The full error provides us some extremely useful context. Something cannot run the command /var/www/vhosts/cfhs69.com/cgi-bin/alumni001A.pl. Glancing at your website I'm going to bet that that command is what is run by Apache to serve the page http://www.cfhs69.com/cgi-bin/alumni001A.pl. Which tells me that the error message is almost certainly coming from Apache, and not your code. Googling around I am willing to bet that it has to do with the problem explained in http://httpd.apache.org/docs/1.3/misc/FAQ.html#fdlim. Namely that your hosting provider is trying to run too many virtual hosts out of Apache, with the result that sometimes you get an Apache process that has opened up too many files, connections, etc and can't open another. (They may be using Apache 2, but the same issue still exists because it is an operating system limit, not an Apache limit.) In which case the possible solutions are that they can raise the operating system limit as is documented there, they can run fewer virtual hosts per server, or you can find a more competent hosting provider. PS: Please call the language Perl, not PERL. See the FAQ for verification. Calling it by the wrong name grates on people who are competent with the language. So unless you actively want to irritate people here, you really should call it by the right name.	[reply]
Re^4: How to debug "Too Many Open Files" Error by kubrat (Scribe) on Sep 17, 2008 at 15:29 UTC
Just to add to the possible solutions bit. Another popular solution is to configure Apache to write to its access and error logs via pipes.	[reply]
Re^5: How to debug "Too Many Open Files" Error by tonyb48 (Novice) on Sep 17, 2008 at 22:35 UTC
I assume this is something I would tell the host, right? Help me understand the suggestion: Is it that Apache is typically configured to write the access and error logs as open files? And if it is reconfigured as you suggest, then these operations would not count as open files? If this be true, what is the downside? I.E., why did he not configure it that way in the first place? I'm just trying to get smart so I can have an intelligent conversation with the host. Thanks Tony	[reply]
Re^4: How to debug "Too Many Open Files" Error by tonyb48 (Novice) on Sep 17, 2008 at 19:52 UTC
Thanks very much for this answer. It will help me have a more constructive dialog with my hosting service. They offered to migrate my sites (I have a second similar site for a different alumni body, which is having the same problems) to a different server with less traffic. I will refer them to the article you referenced. I will ask them to run the commands: `echo 8192 > /proc/sys/fs/file-maxecho && 32768 > /proc/sys/fs/inode-ma +x` [download] `ulimit -a` [download] `ulimit -n` [download] `ulimit -H` [download] I would also like to try inserting warn `ls -l /proc/$$/fd/` . ' '; [download] into the 3 most popular modules in the script. Finally, thanks for the suggestions on not capitalizing Perl. I had no idea. The last thing I want to do is offend the very people from whom I need help.	[reply] [d/l] [select]
Re^3: How to debug "Too Many Open Files" Error by chrism01 (Friar) on Sep 17, 2008 at 01:51 UTC
Are you sure the alumninnnX.pl progs are exiting properly and not hanging?	[reply]
Re^4: How to debug "Too Many Open Files" Error by kubrat (Scribe) on Sep 17, 2008 at 15:24 UTC
That's a good call since you appear to be using CGI. Run top or ps and look for many processes of the same name, usually http or perl or the name of one of your scripts. Moreover if they are not exiting properly they will be marked as defunct.	[reply]