PROCESS: Too many child process, serving the connection

r1n0 has asked for the wisdom of the Perl Monks concerning the following question:

Hello All,
I have a written a proxy server, using HTTP::Proxy, and it has been working fine for 1-10 clients without any problems. I opened the testing up to more clients (upwards of 80), and the program runs fine for a little while and eventually shuts down without any more than the following message:

PROCESS: Too many child process, serving the connection

I am running the script on a linux system, and have HTTP::Proxy setup in the following way:

sub StartServer_tcp_incoming_handler{
    use HTTP::Proxy;
    use HTTP::Proxy::HeaderFilter::simple;
    use Digest::SHA1 qw(sha1_hex);
    my $port = 30000;
    my $proxy = HTTP::Proxy->new( 
            host                    => $Server_IP,
            max_clients             => 80,
            min_spare_servers       => 10,
            max_keep_alive_requests => 1,
            port                    => $Server_Port );
    
    my $filter = HTTP::Proxy::HeaderFilter::simple->new(
        sub {
            my ($self, $headers, $request) = @_;
            my $uri = $request->uri();
            .........
            .........
            ...etc...
[download]

Can I force the TCP connection to be closed after each connection? I think too many open TCP connections are being left open.

Any help is greatly appreciated
Thanks.

Comment on PROCESS: Too many child process, serving the connection Download Code

Replies are listed 'Best First'.
Re: PROCESS: Too many child process, serving the connection by flexvault (Monsignor) on May 02, 2011 at 12:57 UTC
I have never used "HTTP::Proxy", but I did have a similar experience with an application that listened on a port and then forked to process the request. When the requests were coming in for a small number of users, all worked great. So I upped the number of users and after 15-20 minutes the application just crashed. I re-designed the application to pre-fork 4 servers per core, and everything just continues to work without any problem. What I think happened, was that the perl 'fork' function just couldn't keep up the incoming volume of requests. Thank you "Well done is better than well said." - Benjamin Franklin	[reply]
Re: PROCESS: Too many child process, serving the connection by anonymized user 468275 (Curate) on May 02, 2011 at 15:05 UTC
The symptoms occur when new requests are arriving faster than they are being processed, so the excess builds up until a maximum is exceeded. One heavier duty alternative might be Catalyst::Engine::HTTP::Prefork. One world, one people	[reply]