Not exactly sure where to start here:
-
The graph you provided is upside down. Please provide your spreadsheet that you used to generate it, I am very confused how you could even make that happen. You'd have to have multiplied all y values with a -1 to cause such a thing.
I have provided a correctly oriented graph, along with the first derivative here:
Correct Graph
Blue is the source data
Orange is the first derivative
-
The 3 curves presented here are not to scale and lack axes. As such they misrepresent the meaningfulness of the first derivative in peak detection.
-
Your conclusion that upside down derivatives are not useful in peak detection is in fact correct. I assure you that a correctly oriented first derivative is useful for this type of problem.
-
Even though your chart is upside down, your first and second derivatives seem to be calculated correctly (for the upside down data). If you redraw it to scale with axes you will notice that any time the first derivative crosses the x axis (aka it is zero) there will be a peak. This is a very useful property of the first derivative for such peak detection.
With a very small perl program (less than 200 lines) I have a peak detector that works pretty well for this type of psuedo-sinusoidal data. It doesn't take much to do this kind of detection once you have the first derivative calculated and saved.
Given that - as with all solutions, it must fit the problem, and my data is probably a lot different than other people's data (although it's pretty close to the OPs)
| [reply] |
#! perl -slw
use strict;
use Data::Dump qw[ pp ];
use GD;
use constant {
WHITE => unpack( 'N', pack 'CCCC', 0, 255, 255, 255 ),
RED => unpack( 'N', pack 'CCCC', 0, 255, 0, 0 ),
GREEN => unpack( 'N', pack 'CCCC', 0, 0, 255, 0 ),
BLUE => unpack( 'N', pack 'CCCC', 0, 0, 0, 255 ),
};
my( @x, @y, @yd1, @yd2 );
( $x[@x], $y[@y], $yd1[@yd1], $yd2[@yd2] ) = map{ $_ //= 0 } split whi
+le <DATA>;
chomp @yd2;
$_ = ( $_ -4 ) * 1000 for @x;
$_ /= 6 for @y;
$_ = $_ / 320 + 400 for @yd1;
$_ = $_ / 8000 + 400 for @yd2;
my $im = GD::Image->new( 1000, 800, 1 );
$im->filledRectangle( 0, 0, 1000, 800, WHITE );
$im->line( 0, 400, 1000, 400, 0 );
$im->line( $x[$_-1], $y[$_-1], $x[$_], $y[$_], RED ) for 1 .. $#x;
$im->line( $x[$_-1], $yd1[$_-1], $x[$_], $yd1[$_], GREEN ) for 2 .. $#
+x-1;
$im->line( $x[$_-1], $yd2[$_-1], $x[$_], $yd2[$_], BLUE ) for 3 .. $#x
+-2;
my @deltas = map{ ( 0 <=> $y[$_-1] - $y[$_] ) } 1 .. $#y;
$deltas[$_-1] != $deltas[$_] and $im->line( $x[$_], 0, $x[$_], 800, 0
+) for 1 .. $#x-1;
$im->flipVertical;
open PNG, '>:raw', "$0.png" or die $!;
print PNG $im->png;
close PNG;
system 1, "$0.png";
And the dataset taken directly from salva's post above:
With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
| [reply] [d/l] [select] |
Strawman?
Our discussion here is in reply to the parent who made a claim that the second derivative was useful in finding extrema. I have posted so that future readers will not believe this, as I believe it to be untrue. In reply to my claim you have posted a number interesting things - but none seemingly address the claims of the parent, or myself.
The OP claims that the second derivative is useful for determining extrema. His claim suggests that looking at toggling signs on the resulting series indicates extrema. I claim this as *false*.
I also claim that it is possible to find extrema using the first derivative. Any time the derivative crosses the X axis there will be a maxima or minima present within the source series. I do *not* claim that this will find all extrema - I do claim that it will also require additional processing to capture all extrema of interest. Are you claiming this is false?
Your above program is quite a bit shorter than mine. I have a different type of data, but I am pretty sure that looking at deltas like this is a more precise and efficient way of solving the problem than using first derivatives - I disagree that it's more accurate, but that's nitpicking. Overall it is better than doing solely a first derivative and picking from that.
Finally, I strongly disagree that using inverted graphs (even if that's what's used in computer graphics) is a good way to prove your point about much of anything. That said, had you labelled your graph and explained what it was I we both could have saved some time
| [reply] |