Category: | Utility Scripts |
Author/Contact Info | Azhrarn |
Description: | After a day of trying to figure out why one of my web servers was locking up, I found that it was using a bit too much memory. But I had no idea how much, and Linux memory reporting is a bit arcane at best. Especially with something like apache + mod_perl/php using shared memory pools. So after some analysis, I came up with the included script.
It has the following output: Total memory available: 3.21G Total used by apache2 (451 instances): 3.80G Total used by other processes: 0.12G Average memory used per apache2 process: 8.63M Recommended number of processes based on Average: 381 Needed memory for 500 processes based on Average: 4.21G Max memory used for apache2 process: 17.61M Recommended number of processes based on Max: 186 Needed memory for 500 processes based on Max: 8.60G Mean plus two Standard Deviations (bulk of usage under max): 11.28M Recommended number of processes based on Mean + 2*Stdev: 291 Needed memory for 500 processes based on Mean + 2*Stdev: 5.51G At the time I ran it, I had 450 instances out of a configured max of 700 active. Come to find out I only had the ram to support about 380 of those at best. :( Hopefully some other people find this useful, as it's pretty hard to get a straight answer on "How many processes can I support?" Or even quickly analyze how much memory a threaded process with shared memory is using. Note I don't bother to include the root apache process, or the negligible amount of base shared memory. This can also be applied to other arbitrary processes by changing the name it scans for. It is also dependant on the "top" utility, as included in Debian. I'm not sure how it will act under other variations of top, although it should be fairly robust at pulling out the column names. |
#!/usr/bin/perl # Copyright Erik Jacobson - erik@underhanded.org use warnings; use strict; use Statistics::Descriptive; my $webuser = 'www-data'; # The user your apache children run as. I i +gnore the root process. my $command = 'apache2'; # The name of your processes as they appear t +o the "top" command. my @top = `top -bn1`; my %header; my $mem = Statistics::Descriptive::Full->new(); my $othermem = Statistics::Descriptive::Full->new(); my $totalmem; foreach (@top) { s/^\s+|\s+$//sg; my @line = split(/\s+/, $_); if(/mem:\s+(\S+)\s+total/i) { $totalmem = convtok($1); } if(defined $line[3] && exists $header{res} && $line[0] =~ /^\d ++$/) { next unless (defined $line[$header{user}] && defined $ +line[$header{res}] && defined $line[$header{shr}] && defined $lin +e[$header{command}]); if(($line[$header{user}] eq $webuser) && ($line[$heade +r{command}] eq $command)) { $mem->add_data(convtok($line[$header{res}]) - +convtok($line[$header{shr}])); } else { $othermem->add_data(convtok($line[$header{res} +]) - convtok($line[$header{shr}])); } } elsif (!exists $header{res}) { my %tempheader; for (0 .. $#line) { $tempheader{lc($line[$_])} = $_; } if(defined $tempheader{user} && defined $tempheader{re +s} && defined $tempheader{shr} && defined $temphe +ader{command}) { %header = %tempheader; } } } printf "Total memory available: %.2fG\n", ($totalmem / 1024 / 1024); printf "Total used by $command (%d instances): %.2fG\n", $mem->count() +, ($mem->sum() / 1024 / 1024); printf "Total used by other processes: %.2fG\n\n", ($othermem->sum() / + 1024 / 1024); $totalmem -= $othermem->sum(); printf "Average memory used per $command process: %.2fM\n", ($mem->mea +n() / 1024); printf "Recommended number of processes based on Average: %d\n", ($tot +almem / $mem->mean()); printf "Needed memory for 500 processes based on Average: %.2fG\n\n", +($mem->mean() / 1024 / 1024 * 500); printf "Max memory used for $command process: %.2fM\n", ($mem->max() / + 1024); printf "Recommended number of processes based on Max: %d\n", ($totalme +m / $mem->max()); printf "Needed memory for 500 processes based on Max: %.2fG\n\n", ($me +m->max() / 1024 / 1024 * 500); printf "Mean plus two Standard Deviations (bulk of usage under max): % +.2fM\n", (($mem->mean() + $mem->standard_deviation() * 2) / 1024); printf "Recommended number of processes based on Mean + 2*Stdev: %d\n" +, ($totalmem / ($mem->mean() + $mem->standard_deviation() * 2)); printf "Needed memory for 500 processes based on Mean + 2*Stdev: %.2fG +\n", (($mem->mean() + $mem->standard_deviation() * 2) / 1024 / 1024 * + 500); sub convtok { my $num = shift; if($num =~ /^\s*([0-9.]+)k\s*$/i) { $num = $1; } elsif ($num =~ /^\s*([0-9.]+)m\s*$/i) { $num = $1 * 1024; } elsif ($num =~ /^\s*([0-9.]+)g\s*$/i) { $num = $1 * 1024 * 1024; } return $num; } |
|
---|
Replies are listed 'Best First'. | |
---|---|
Re: Analyzing Apache memory usage
by zentara (Cardinal) on Jul 18, 2008 at 15:31 UTC | |
Re: Analyzing Apache memory usage
by Anonymous Monk on Sep 22, 2020 at 09:36 UTC | |
Re: Analyzing Apache memory usage
by Anonymous Monk on Oct 29, 2011 at 20:25 UTC | |
Re: Analyzing Apache memory usage
by Anonymous Monk on Oct 29, 2011 at 19:55 UTC | |
by Anonymous Monk on Oct 29, 2011 at 20:20 UTC | |
Re: Analyzing Apache memory usage
by Anonymous Monk on Oct 29, 2011 at 20:10 UTC | |
by Anonymous Monk on Oct 29, 2011 at 20:21 UTC |