Re^7: using Statistics::Regression

It's not the amount of change that is the problem, it's the number of distinct values. In your last example, there are only 2 different values of x (2.5 and 3.25), so you can't get more than a linear fit. This is a fundamental limitation of the underlying mathematics.

Comment on Re^7: using Statistics::Regression

Replies are listed 'Best First'.
Re^8: using Statistics::Regression by Random_Walk (Prior) on Apr 20, 2017 at 14:52 UTC
Thank you, I see what you mean now. I guess I expected to get theta of zero for all the terms but the constant, if there was not enough data to get more than a line fit. I have now updated the code to count the number of unique values and never ask for too many terms. Script now runs happily for all my data. The predictions are still a little wild though, I suspect I may be over-fitting. I don't suppose you saw any undocumented features to add a regularisation term in the code :) Cheers, R. Pereant, qui ante nos nostra dixerunt!	[reply]

Replies are listed 'Best First'.

Re^8: using Statistics::Regression
by Random_Walk (Prior) on Apr 20, 2017 at 14:52 UTC

Thank you, I see what you mean now.

I guess I expected to get theta of zero for all the terms but the constant, if there was not enough data to get more than a line fit. I have now updated the code to count the number of unique values and never ask for too many terms. Script now runs happily for all my data. The predictions are still a little wild though, I suspect I may be over-fitting. I don't suppose you saw any undocumented features to add a regularisation term in the code :)

Cheers,
R.

Pereant, qui ante nos nostra dixerunt!

[reply]