in reply to Re: Any idea for predicting the peak points in the graph by perl
in thread Any idea for predicting the peak points in the graph by perl

This node falls below the community's threshold of quality. You may see it by logging in.
  • Comment on Re^2: Any idea for predicting the peak points in the graph by perl

Replies are listed 'Best First'.
Re^3: Any idea for predicting the peak points in the graph by perl
by kgraff (Monk) on Apr 18, 2005 at 14:01 UTC

    Use a second derivative test (from positive to negative) to find the peaks. For valleys, the second derivative will go from negative to positive.

    Finding peaks on chromatograms, spectra, Flow Injection charts, etc. is a fairly common operation in laboratories. There are also a lot of cookbook solutions, like running standard deviations for determining noise levels. Are you "rolling your own" for practice?

Re^3: Any idea for predicting the peak points in the graph by perl
by salva (Canon) on Apr 18, 2005 at 16:19 UTC
    have you tried solving your problem on a spreadsheet (Excel or Calc from OO.org) first?

    Insert the data on a table and add colums for the first and the second derivates (you can find some formulas to do it below), then create an X-Y graph showing Y, Y' and Y'' so you could visualize and understand what other monks have suggested you about looking for changing signs on the second derivates.

    Coding a subrutine in perl to solve the problem and others similar will be much easier then.

    X Y Y'=(Y[n+1]-Y[n-1])/(X[n+1]-X[n-1]) Y''=(Y'[n+1]-Y'[n-1])/(X[n+1]-X[ +n-1]) 4.1330 4580.87 4.1470 4522.75 -1328.56 4.1600 4545.00 -7791.65 -935707.98 4.1730 4320.17 -25656.96 -1544908.57 4.1870 3852.26 -49504.19 -1396364.62 - 4.2000 2983.56 -63358.81 59518.52 + 4.2130 2204.93 -47956.7 1368481.22 4.2270 1688.73 -26409.81 1302338.03 4.2400 1491.87 -12793.58 687964.39 4.2530 1356.09 -8522.74 246232.62 4.2670 1261.75 -6145.3 120543.1 4.2800 1190.17 -5268.08 99532.76 4.2930 1124.78 -3557.44 180758.68 4.3070 1094.12 -387.59 357928.14 4.3200 1114.32 6106.62 533914.53 4.3330 1252.89 13494.19 473482.01 4.3470 1478.66 18890.63 334228.18 4.3600 1762.94 22518.35 61078.35 + 4.3730 2064.14 20478.67 -278669.88 - 4.3870 2315.86 14994.26 -533957.74 4.4000 2468.98 6061.81 -715688.03 4.4130 2473.47 -3613.63 -574250.77 4.4270 2371.42 -9442.96 -274812.86 - 4.4400 2218.51 -11033.58 93502.85 + 4.4530 2084.54 -7011.89 376399.97 4.4670 2029.19 -870.78 468240.9 4.4800 2061.03 5630.62 455004.27 4.4930 2175.58 10959.33 278503.96 4.5070 2356.93 13150.22 13708.45 + 4.5200 2530.64 11329.46 -316569.8 - 4.5330 2651.50 4919.41 -526292.81 4.5470 2663.46 -2880.44 -563985.17 4.5600 2573.73 -10308.19 -476770.66 4.5730 2395.45 -15276.48 -346338.56 4.5870 2161.26 -19659.33 -286611.8 4.6000 1864.65 -23015 -18659.54 - 4.6130 1562.87 -20144.48 218052.13 + 4.6270 1320.75 -17127.59 195150.31 4.6400 1100.43 -14875.42 287534.19 4.6530 933.99 -9651.7 387333.91 4.6670 839.83 -4417.41 369091.59 4.6800 814.72 313.77 341683.76 4.6930 847.99 4466.37 298057.93 4.7070 935.31 8361.33 382671.47 4.7200 1073.75 14798.5 519444.44 4.7330 1320.07 21866.89 519764.75 4.7470 1664.15 28832.15 506969.93 4.7600 2098.54 35555.08 4.7730 2588.58

      I was searching for a pre-made perl solution for performing peak-detection when I stumbled upon this post.

      To anyone following in my footsteps know that the OP incorrectly represents information regarding the meaning of derivative and second derivatives.

      Specifically the second derivative is not required nor necessarily useful in assisting with peak detection.

      Second derivatives represent change to the rate of change of a series (in this context). Peak detection has little to nothing to do with such change to the rate of change (aka acceleration). Instead we are more interested in the first derivative, which is more analogous to rate of change of the originating series (aka velocity).

      While performing any peak detection your goal is to capture the points in a series whereby the point has reached a local maximum or minimum. One can guesstimate that if velocity is not changing, and then subsequently reverses (changes sign) that this would likely represent a peak of some type. Further analysis using the series mean and standard deviation can help determine if this is in fact a meaningful peak to you or not (it's all circumstantial depending on your needs).

      So the OP's suggestion that observing changing signs on the second derivative is not correct. Additionally while observing sign changes on the first derivative may prove useful, you will need to perform additional analysis to determine if it's the exact data you are looking for.

        And for those that follow you. Neither the first nor second derivative is a particularly good indicator of peaks and troughs. At least not in this data.

        In this graph, red is the original data; green the first derivative and blue the second.

        Note that the green lines inflection points don't track the reds in anyway.

        The second derivative has inflections corresponding to those in the data. But it also "invents" new ones where they do not exist.

        And can anyone automate the determination of which of the 5 or 6 reversals in the last segment of the blue line corresponds to the highest peak in the original data?


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

        The start of some sanity?

Re^3: Any idea for predicting the peak points in the graph by perl
by QM (Parson) on Apr 18, 2005 at 14:40 UTC
    You need to keep track of the sign of the y-delta of the last pair, and compare it to the sign of the y-delta of the current pair.

    The following is a start. Input is not validated. Adjacent "peak" points with the same y value will be skipped -- you'll need to add some code for that. "Peak" points that occur on the first or last data point will be skipped. There may be an error on the first pair with some input.

    Untested:

    #!/your/perl/here use strict; use warnings; my $last_sign; my $last_y; my $last_x; while (<>) { my ($x,$y) = split; my $sign = $y <=> $last_y; if ( $last_sign <=> $sign ) { print "$last_x $last_y\n"; } $last_x = $x; $last_y = $y; $last_sign = $sign; }
    Have the Homework Police cleared this yet?

    -QM
    --
    Quantum Mechanics: The dreams stuff is made of

Re^3: Any idea for predicting the peak points in the graph by perl
by tlm (Prior) on Apr 18, 2005 at 15:14 UTC

    Here's a generalization of a snippet I posted earlier (which, BTW, had a my bug, now fixed); it does both maxima and minima:

    use strict; my $input_file = 'foobar.txt'; open my $in, $input_file or die "Failed to read $input_file: $!\n"; my @max = do { local $_ = <$in>; chomp; [ split ] }; my @min = ( [ @{ $max[ 0 ] } ] ); my $last = $max[ 0 ][ 1 ]; while ( <$in> ) { chomp; my ( $x, $y ) = split; if ( $y > $last ) { $max[ -1 ] = [ $x, $y ]; } elsif ( $max[ -1 ] ) { $max[ @max ] = undef; } if ( $y < $last ) { $min[ -1 ] = [ $x, $y ]; } elsif ( $min[ -1 ] ) { $min[ @min ] = undef; } $last = $y; } close $in; pop @max unless $max[ -1 ]; pop @min unless $min[ -1 ]; print "Maxima:\n"; print "($_->[ 0 ], $_->[ 1 ])\n" for @max; print "Minima:\n"; print "($_->[ 0 ], $_->[ 1 ])\n" for @min; __END__ Maxima: (4.133, 4580.870000) (4.160, 4544.999000) (4.413, 2473.469000) (4.547, 2663.464000) (4.773, 2588.583000) Minima: (4.147, 4522.753000) (4.307, 1094.119000) (4.467, 2029.188000) (4.680, 814.717000)

    the lowliest monk