Re: Excel or Perl for simple statistics?

There's a difference, IIRC, between the population standard deviation and the sample standard deviation (again IIRC, this is sometimes called standard error - I'm sure a statistician will explain the issues). Excel has functions for both - stdev and stdeva. Are you sure which you want to use and that you are using the equivalent in both?

Regards,

John Davies

Update: I didn't mention stdevp as I didn't know about it. Thanks, poj.

Comment on Re: Excel or Perl for simple statistics?

Replies are listed 'Best First'.
Re^2: Excel or Perl for simple statistics? by Anonymous Monk on Jan 20, 2018 at 16:10 UTC
Yes, I also see the standard error but this is like 0.0896196283 or something, nothing close to 0.41 or 0.401. My question is, would you use the Perl module or just stick with Excel?	[reply]
Re^3: Excel or Perl for simple statistics? by pryrt (Abbot) on Jan 20, 2018 at 17:36 UTC
I believe davies was wrong about it also being called the standard error: according to wp, Standard_error, the standard error is the standard deviation divided by the root of the number of samples. The population standard deviation (Excel: `=STDEVP()` or `=STDEV.P()`) assumes you that have data for the entire population; the sample standard deviation (Excel: `=STDEV()` or `=STDEV.S()`) assumes that your `n` data points are just a sample of the population, so it uses a different denominator (`n-1`) inside the radical, to account for the uncertainty because it's just a sample, not the whole population. Given your set of data, the 0.401791 is the population standard deviation, and the 0.410689 is the sample standard deviation. Population standard deviation: σ = sqrt( Σ{ (x_i - x_avg)² } / { n } ) Sample standard deviation: s = sqrt( Σ{ (x_i - x_avg)² } / { n - 1 } ) According to Statistics::Basic, you can set `$Statistics::Basic::UNBIAS` true to get the sample, or (implied) set it false to get the population. Statistics::Lite directly says that their `stddev()` function is the unbiased (sample) n-1, which matches the Excel `=STDEV()` function, and they have the `stddevp()` for the population n, matching the Excel `=STDEVP()`.	[reply] [d/l] [select]
Re^3: Excel or Perl for simple statistics? by Anonymous Monk on Jan 20, 2018 at 16:38 UTC
Ok, so problem seems to be fixed if I use `Statistics::Lite` module instead... Do you think it could be associated to the fact that with `Statistics::Lite` I pass the numbers as array instead of array reference that I was using in `Statistics::Basic` My list was: `0.964 0 0 0.94 0.895 0.915 0.775 0.868 0 0.796 0.866 0.819 0 0 0 0 0.794 0.806 0.781 0.807 0.783` [download]	[reply] [d/l] [select]
Re^4: Excel or Perl for simple statistics? by poj (Abbot) on Jan 20, 2018 at 17:06 UTC
.. as array instead of array reference .. I get the same either way and no difference with Excel `#!perl use strict; use Statistics::Basic 'stddev'; my @data = (0.964, 0, 0, 0.94, 0.895, 0.915, 0.775, 0.868, 0, 0.796, 0.866, 0.819, 0, 0, 0, 0, 0.794, 0.806, 0.781, 0.807, 0.783); print stddev(\@data)+0; print "\n"; print stddev(@data)+0; # 0.400791162022762 # excel # STDEVP # Calculates standard deviation based # on the entire population given as arguments. # 0.400791162` [download] poj	[reply] [d/l]