in reply to mod_perl go boom, mod_cgi works

You haven't mentioned which version of perl you're using. I've noticed memory problems that went away when upgrading from 5.6.x to 5.8.x. Specifically, I was using XML::Twig, which, of course, uses XML::Parser, which I'm sure at least some of your modules use. I suppose the overhead in all the XML handling that I was doing managed to confuse 5.6, but those problems were fixed in 5.8.

So, that's my first offering. My experience managed to crash in inconsistant locations, but, abstractly it was consistant: every time I ran my program. My program was taking an existing ~50KB XML file, and then add data it got from another source to make a multi-hundred KB XML file (actually, I have a hard time remembering, but I'm guessing it was multi-MB at the end). This may be your problem in that, over multiple requests, you're chewing up too much memory - and some of it may be memory junked inside the expat XML parsing C library.

Thus, one solution is to upgrade perl, if you haven't already gone to perl 5.8.3+. Another is to reduce the number of requests each Apache child can handle before exiting, thus exiting before the accumulation of problem memory chunks to the point of a crash. This latter idea is not dissimilar to the idea of rebooting NT4 servers nightly to keep them from crashing after about 40 hours of uptime, which I seem to recall was a popular problem. It's not the right solution (the server should be fixed - in this case, it's probably between Perl and Expat), but it offers a useful workaround until then.

Replies are listed 'Best First'.
Re: mod_perl go boom, mod_cgi works
by hacker (Priest) on Mar 06, 2005 at 15:48 UTC
    "Another is to reduce the number of requests each Apache child can handle before exiting, thus exiting before the accumulation of problem memory chunks to the point of a crash."

    This idea strikes me in two ways...

    1. The code or running application is not freeing memory or globals at each iteration...
    2. This is just a workaround for badly-written code that is incorrectly using globals or some other shared structures.

    That being said, I've limited MaxRequestsPerChild in Apache to 2, and so far, I am not able to reproduce the problem. I'm not confident this "solved" the problem however...

    One thing I did notice though, was that when I'm clicking through my sample feeds linked from the front page of this mod_perl application, sometimes my click for 'foo.bar.xml' will report that 'example.com.rdf' does not exist. example.com.rdf is a feed I might have clicked last, or 10 clicks ago. It seems random, but leads me to believe there is some persistence going on here. A clue?

    Is there a more detailed way to find out if I'm using globals somewhere, that my old eyes can't see?

Re: mod_perl go boom, mod_cgi works
by hacker (Priest) on Mar 06, 2005 at 14:05 UTC
    Thanks for the suggestion.

    Unfortunately, I'm already running 5.8.4, and I've taken the time to rebuild cleanly, every single module used in this chain, from upstream latest available sources. I was specifically pedantic about making sure EVERY test passed with 100%.