in reply to Average over time
There are all sorts of reasons why averaging over a time period is a bad way to achieve what you want. The fact you have no option but to keep all the values is just one issue. There are also problems of aliasing and delayed estimates.
As kvale said some kind of decaying average would let you work on summary data rather than keeping records.
Have a look at this and this for a more rigerous explination of the issues.
I think what you want to do is have a "Single Exponentially Smoothed" estimate of the average. As I recall this means you should estimate the value for each time period as:
$a_factor = 0.1 # You have to play with different # values of the smoothing factor to # match the data you have $new_avg = $this_period_avg*$a_factor + $prev_avg * (1 - $a_factor);
An $a_factor value of 0.1 is equivalent to a moving average window of something like 20 time slots. So this lets you do the analysis while only maininging 5% of the data (I am sure that some other more expert brother will be able to correct this obvious simplification).
|
|---|