simak has asked for the wisdom of the Perl Monks concerning the following question:

Good day Monks

So, i have CGI script running under apache. Some pages loads fine, but some loads from time to time.

The pages which loads from time to time is generally generates with big, huge script functions which doing some work before prints something in html(but there is no function that working more than 2.5 seconds, so the timeout between printing in html is not bigger than that(maybe this info helps somehow))

When these pages are not loaded i have got messages in httpd.log like these

Mon Mar 22 08:57:39 2010] [info] [client 62.176.11.182] (104)Connection reset by peer: core_output_filter: writing data to the + network [Mon Mar 22 08:57:44 2010] [info] [client 62.176.11.182] (104)Connection reset by peer: core_output_filter: writing data to the + network [Mon Mar 22 08:57:57 2010] [info] [client 62.176.11.182] (104)Connection reset by peer: core_output_filter: writing data to the + network [Mon Mar 22 08:57:59 2010] [info] [client 62.176.11.182] (32)Broken pi +pe: core_output_filter: writing data to the network [Mon Mar 22 08:57:59 2010] [info] [client 62.176.11.182] (32)Broken pi +pe: core_output_filter: writing data to the network [Mon Mar 22 08:58:09 2010] [info] [client 62.176.11.182] (32)Broken pi +pe: core_output_filter: writing data to the network [Mon Mar 22 08:58:26 2010] [info] [client 62.176.11.182] (32)Broken pi +pe: core_output_filter: writing data to the network [Mon Mar 22 09:01:03 2010] [info] [client 62.176.11.182] (104)Connection reset by peer: core_output_filter: writing data to the + network [Mon Mar 22 09:01:05 2010] [info] [client 62.176.11.182] (104)Connection reset by peer: core_output_filter: writing data to the + network [Mon Mar 22 09:01:13 2010] [info] [client 62.176.11.182] (104)Connection reset by peer: core_output_filter: writing data to the + network [Mon Mar 22 09:02:45 2010] [info] [client 62.176.11.182] (32)Broken pi +pe: core_output_filter: writing data to the network [Mon Mar 22 09:02:45 2010] [info] [client 62.176.11.182] (104)Connection reset by peer: core_output_filter: writing data to the + network [Mon Mar 22 09:03:00 2010] [info] [client 62.176.11.182] (32)Broken pi +pe: core_output_filter: writing data to the network [Mon Mar 22 09:03:02 2010] [info] [client 62.176.11.182] (32)Broken pi +pe: core_output_filter: writing data to the network [Mon Mar 22 09:03:06 2010] [info] [client 62.176.11.182] (32)Broken pi +pe: core_output_filter: writing data to the network [Mon Mar 22 09:03:12 2010] [info] [client 62.176.11.182] (32)Broken pi +pe: core_output_filter: writing data to the network

and so on. The apache is in debug mode, so this messages as all i have. I do not think that it is apache problem. I searched through the internet about these messages, there are a lot of people who has this, but i can`t find solution for me. I have Fedora 12, apache 2.2.14, perl 5.10.

Maybe somebody knows what is going on, because i have no more thoughts, i am newbie in all this, so i think i did not try all ideas to understand where is the problem. I repeat that i have no problem with little and lightweight pages, which only prints html(by same script) or doing little more, but not as big work as the script which generates pages on which everything stops with errors above.

Replies are listed 'Best First'.
Re: CGI page sometimes do not loading, apache write error
by almut (Canon) on Mar 24, 2010 at 06:48 UTC

    As the error is happening when Apache is trying to write output to the network connection (browser), it is unlikely that this has to do with the CGI script's content generation.  Common reasons for this info message are a flaky network connection, the user at the browser being impatient and pressing Esc before the page has finished loading, etc.

    What isn't clear to me from your description: are you actually sitting at the browser watching the connection being cut without you doing anything (i.e. the page failing to load), or are you just seeing those messages in your log file without knowing what's happening at the user/browser side?

      Yes - i am watching it myself from browser, so i know what is going on at the browser side. Page begins to load, generation is going on(there are many divs and imgs) and it can stop at any moment. Stops means no to get the end(</html>) of loading page, but it seems just like the content stops to generate.

      P.S. It is strange that sometimes it loads entire the page(in some cases it takes 5-10 minutes) without problems and sometimes it stops loads just after 2 seconds of loading the same page

        According to the log messages, the connection is being closed client-side, i.e. by the browser, a proxy, or whatever other network problem in between apache and the browser.  So that's what you'd have to focus on.

        You could try a commandline tool such as wget, or LWP::UserAgent or lwp-dump from libwww-perl, etc. to see whether you then get the entire html (without images).  Maybe the loading of the many images is somehow exceeding some local resource limit, so something in between (proxy, network monitoring?) is terminating the request(s)...  Just a guess.

        BTW, what browser are you using? Have you tried a different one?  Also, is this happening systematically or sporadically (with the same page)?

        Let me guess, its a big page? Write output to temporary file, gzip it, redirect user to that url, that way the CGI won't timeout before the users slow connection is able to read the file.
Re: CGI page sometimes do not loading, apache write error
by simak (Sexton) on Mar 27, 2010 at 12:49 UTC

    After this:

    So, i made some tests. I generated 200 pages local - right on the server. perl simashin.cgi params > file_name.html

    The result is : * Each of 200 pages generates till the end -> </html> without any errors. So we can make conclusion, that the cgi script works fine * This static generated pages can easily load with browser. So we make the conclusion that problems not in content of page.

    So the problem can be only in connection(provider, home server, router, something that i can`t even imagine :) ...) or in apache

    Thinking about - how can i find the weak place I think it is time to some tests with LWP

    I wrote simple LWP script, which getting this problem page -

    #!/usr/bin/perl use HTTP::Request::Common; require LWP::UserAgent; use Data::Dumper; $ua = new LWP::UserAgent; $ua -> timeout(60); $request = new HTTP::Request ('GET',"http://sitename/script.cgi?parame +ter1=value1&parameter2=value2"); $response = $ua -> request($request); print Dumper($response);

    I exec this script in cycle like this

     for i in 1..n; do test.pl > test_$i.html; done

    You can do it as you like(right in perl script maybe)

    I did something about 200 test, and after it making some changes in network,apache,router etc... settings, and analyzing the result of execution.

    So i had only ~5% of pages loaded to the end, after day of testing. So i decided to test apache(doing generation of this page) i edit the test script and wrote there localhost instead of full domain name, and run it local. The apache works good without mistakes, after couple of hours i saw that 99% of pages was generated to the end... So apache works fine.

    So the problem now can only be in network or network devices.I thought on my provider first, but my network connection seems to be good - i have 2-3ms ping and always 10mbit in each direction connection.

    So i begin to trace network, running tcpdump, making loads with telnet, trying to analyze errors and cases when page stops to download, but can find nothing.

    Network adapters seems works fine too, so i decide to try connection without my router - dlink dir300(which gives Internet to 3 +wifi computers), the 1 day gone while i made my linux server work with my provider connection type :D, they(provider) said that it can be done only through a couple numbers of router or through windows special program, but it is not true, as always, now i know that i can do it from linux :D. Arrggghh, about the problem - what do you think - it is works fine...

    So the problem is in this router. Moreover when this errors begins the first problem device on which i thought - was this router(ugly little box :) ), because router is closed device(it has only web interface for settings) and you can`t understand working it or not, you can`t to know how it actually works, what is the load of it cpu and so on. But i did not want to test the connection without it, because of hardness to configure linux server with my provider connection.

    After understanding where is the weak place i gather home network back(with router), and begin to make some tests, so the problem was - when the redirection of ports working on router, we have only 5% good results with this big script, so i remove the redirection for my linux server(but now it is all open in internet :( ), and now all works fine

    P.S. So, from time i begin a linux user and have linux sysadmin work, i begin to hate these closed boxes on which you do not have the full control, but this is progress and for usual users it is great devices, which easily can be configured... More over if we try to control all things ourselves in our life we will be crazy :D

    P.S.2. Thanks to all who helps me.

    P.S.3. I hope this post can help all other who searching the solution for this apache errors with cgi