in reply to Finding local maxima/minima in noisy, pediodic data

Push the data through a low-pass filter: 
use constant {
    alpha   => 0.1,
};

my ($prev_x, $prev_y) = (0, 0);
while (<>) {
    my ($t, $x, $y) = split;

    $x = $prev_x + alpha * ($x - $prev_x);
    $prev_x = $x;

    $y = $prev_y + alpha * ($y - $prev_y);
    $prev_y = $y;

    print "$t $x $y\n";
}

[download]
Tweak the alpha to get at the desired frequency band.

Replies are listed 'Best First'.
Re^2: Finding local maxima/minima in noisy, pediodic data
by Anonymous Monk on Oct 12, 2009 at 22:09 UTC
    If I plot your test data up, there is a wave with a period of something like 3000, and there is a wave with a period of about 60. Is it the long period you are looking for extrema in?
[reply]
Re^2: Finding local maxima/minima in noisy, pediodic data
by Anonymous Monk on Oct 13, 2009 at 00:22 UTC
    If I assume that it is the 3000 period wave you are looking for extrema in, I can see you keeping track of 2 quantities. One is the long term average, something which takes in more than 2 of the long term waves (hopefully much more than 2). The other is a moving average, of about 1/3 the period (so about 1000 points). Occasionally a person has to calculate the average of that 1000 points, but for many iterations it is probably easier to just use a circular buffer of all the points, and to calculate the new average you subtract out the value to be replaced and add in the new value. If the 1000 term MA is below the long term average, you are looking for minima. If the 1000 term MA is above the long term average, you are looking for a maxima.
[reply]
      No, I'm interested in the extrema of the short wavelength pulses.

      However, I can use your suggestion about long term averages for that, too.

      A possible strategy: In the first pass I generate a smoothed dataset from the original using a moving average of about 200 points (or 2-3 periods of the short pulses). This will smoothen out the short wavelength pulses but retain the long term characteristics. Then I compare the original with this smoothed dataset point by point: if it is below the average I need to look for a minimum, if it is above, I look for a maximum.
[reply]

        With any sort of filter (moving average or low pass) you will induce a delay on the response. That is to say, if you're at a peak in the filtered data and want to match it with the original data, you will need to look back a fixed period of time.

        For what it's worth, I think the low pass filter will be much more applicable to your goal since the noise is high frequency periodic. The alpha I provided above does a fine job smoothing out the noise but leaving the pulses of the desired frequency.

[reply]

In Section Seekers of Perl Wisdom