Re^3: leak detection in other (non-perl) processes

Replies are listed 'Best First'.
Re^4: leak detection in other (non-perl) processes by apprentice (Scribe) on Mar 23, 2012 at 19:53 UTC
OK, let's say that I want to monitor process X, which could be affected by repeatedly killing process Y. That is, I am attempting to 'induce' leaks. So, I get a baseline, like say '17' open file descriptors (FDs) for process X. Then I loop for 'n' iterations, killing process Y, waiting a bit, getting current FD count, storing it, next. Now, I may have a list that looks like one of the following: For example, lets use file descriptors, here are some lists of possibl +e values A = 17 17 17 17 17 17 17 17 17 17 17 17 17 17 17 17 17 17 17 17 17 17 +17 17 B = 17 17 17 17 17 17 17 17 17 17 17 17 16 17 17 17 17 17 17 17 17 17 +17 17 C = 17 17 17 17 17 17 17 17 17 17 17 17 17 17 17 17 17 17 17 17 17 17 +17 25 D = 17 20 17 20 17 20 17 20 17 20 17 20 17 20 17 20 17 20 17 20 17 20 +17 20 E = 17 20 34 29 36 22 18 25 19 19 19 43 52 37 28 19 17 17 17 20 17 27 +36 47 F = 17 20 23 26 29 32 35 38 41 44 47 50 53 56 59 62 65 68 71 74 77 80 +83 86 G = 17 20 20 26 25 26 30 34 38 41 44 39 42 46 50 58 52 55 55 55 55 56 +59 58 Here are some of the thoughts I have jotted down: Example A: not a leak, easy to determine, never changes Example B: not a leak, almost never changes, value only ips for a moment, may reset low value, but does not reset average value (int), so does not represent a leak, easy to determine Example C: does not appear to be a leak, almost never changes, only appears at end of data, low enough value should not change average value much, would statistics help here, to show it as an outlier? may be able to use position as an indicator, no change ever until final position out of n positions would show it to be an outlier (however, a single change earlier that did not reset could be a very slow leak, potentially, requiring more time to evaluate) NOTE: a single change anywhere, that resets, is not a leak Example D: not a leak, changes regularly, up and down the same amounts, harder to detect, can notice that the current number repeatedly drops to the low number (or thereabouts) and the current number only climbs to the max number or thereabouts (can keep average low and average max as well as running average), maybe keep counts of individual numbers or 'buckets' as well, but still possible to determine NOTE: Related, but IS a leak of some sort, would be when it always drops back to the low or thereabouts, but always or evetually always climbs to a higher max Example E: not a leak, changes regularly, but by varying amounts, ends on a high note, so would have to be mindful that it drops back down to or near the low frequently (frequency/counts) and recently (positional) Example F: IS a leak, changes regularly, always up, but by varying amounts, easy to detect (can also count increases, decreases if it helps the other cases) Example G: IS a leak, changes regularly, not every time, sometimes decreases, but relatively few decreases, continued upward trend NOTE: Should continually check that the PID of each process does not change, else the values are no longer any good!!!in which case, they will almost certainly reset to lower values, perhaps climbing again Other considerations: Want to consider the slope, percentage-wise as well as absolute...??? [download] "Peace, love, and Perl...well, okay, mostly just Perl!" --me Apprentice	[reply] [d/l]
Re^5: leak detection in other (non-perl) processes by BrowserUk (Patriarch) on Mar 23, 2012 at 20:52 UTC
Maybe something like this will get you started. Try playing with the threshhold and moving average period (see examples): #! perl -slw use strict; use List::Util qw[ sum min max reduce ]; my %cases = ( A => [ qw[ 17 17 17 17 17 17 17 17 17 17 17 17 17 17 17 17 17 17 1 +7 17 17 17 17 17 ] ], B => [ qw[ 17 17 17 17 17 17 17 17 17 17 17 17 16 17 17 17 17 17 1 +7 17 17 17 17 17 ] ], C => [ qw[ 17 17 17 17 17 17 17 17 17 17 17 17 17 17 17 17 17 17 1 +7 17 17 17 17 25 ] ], D => [ qw[ 17 20 17 20 17 20 17 20 17 20 17 20 17 20 17 20 17 20 1 +7 20 17 20 17 20 ] ], E => [ qw[ 17 20 34 29 36 22 18 25 19 19 19 43 52 37 28 19 17 17 1 +7 20 17 27 36 47 ] ], F => [ qw[ 17 20 23 26 29 32 35 38 41 44 47 50 53 56 59 62 65 68 7 +1 74 77 80 83 86 ] ], G => [ qw[ 17 20 20 26 25 26 30 34 38 41 44 39 42 46 50 58 52 55 5 +5 55 55 56 59 58 ] ], ); our $V //= 0; our $THRESHHOLD //= 1.1; our $MOVING //= 5; for my $case ( sort keys %cases ) { my $v = $cases{ $case }; my @moving = map{ sprintf "%.2f", sum( @{ $v }[ $_ .. $_+$MOVING ] ) / $MOVING } 0 .. $#$v - $MOVING; my $min = min @moving; my $max = max @moving; my $leak = 'no leak'; if( $moving[ 0 ] == $min and $moving[ -1 ] == $max and ( $min * $THRESHHOLD ) < $max ) { $leak = 'a leak '; } printf "Case $case has %s %s\n", $leak, $V ? "[ @moving ]" : ''; } __END__ [20:34:52.28] C:\test>junk47 -THRESHHOLD=1.10 -MOVING=5 Case A has no leak Case B has no leak Case C has no leak Case D has no leak Case E has no leak Case F has a leak Case G has a leak [20:35:07.79] C:\test>junk47 -THRESHHOLD=1.07 -MOVING=5 Case A has no leak Case B has no leak Case C has a leak Case D has no leak Case E has no leak Case F has a leak Case G has a leak [20:35:16.82] C:\test>junk47 -THRESHHOLD=1.07 -MOVING=15 Case A has no leak Case B has no leak Case C has no leak Case D has no leak Case E has no leak Case F has a leak Case G has a leak [20:35:25.46] C:\test>junk47 -THRESHHOLD=1.03 -MOVING=15 Case A has no leak Case B has no leak Case C has no leak Case D has no leak Case E has no leak Case F has a leak Case G has a leak [20:35:32.66] C:\test>junk47 -THRESHHOLD=1.025 -MOVING=15 Case A has no leak Case B has no leak Case C has a leak Case D has no leak Case E has no leak Case F has a leak Case G has a leak [download] With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. The start of some sanity?	[reply] [d/l]
Re^5: leak detection in other (non-perl) processes by Eliya (Vicar) on Mar 23, 2012 at 20:17 UTC
As long as you don't set a fixed time frame for the observation/analysis, everything will be just a more or less useful heuristic. Anyhow, I think you want to fit a trend line through the entire data¹, and if it isn't flat, you likely have a leak. This assumes the evaluation interval is (considerably) longer than the period of any cyclic behavior you might observe. ___ ¹ if you're worried about outliers, you might want to look for a statistics package that can compute the fit based on minimizing linear deviations, instead of squared deviations, as done in a standard "least squares" fit (see also LAD).	[reply]