I'm growing interested in curve fitting some data, and looking at the tools available in Perl. Most aren't robust enough for anything serious, or they lack any kind of error analysis, but PDL::Fit::LM seems a pretty serious bit of code. It returns a covariance matrix, and I can get a standard error from the square root of the diagonal terms of the matrix, and can calculate a cross correlation coefficient from the off diagonal terms divided by the square root of the product of the two relevant diagonal terms.

What I don't recall are some of the subtleties of getting error limits on the fitted parameters. Michael L. Johnson of the Biophysics Department of the University of Virginia was big on this; big on analysis of residuals and calculating true confidence intervals of fitted parameters (in no small part because fitted parameters of real world problems are often highly correlated).

Anyway, the specifics of doing these kinds of analyses escape me presently and I'm not that close to a research library (if I were, i'd just walk down Michael's publication list and find what I need). Has someone turned this kind of "stuff" into a Perl module? If not, what gotcha's are out there? I worked in biophysical chemistry long ago. Issues peculiar to the models I used may not be an issue in other unrelated fields.

Updated: links Further Updated:

A deeper comparison and test of PDL::Fit::LM has revealed issues. If you take the test code provided on CPAN, and reduce the value of the Y data by a factor of 10, the covariance matrix does not change. If you take the test code provided by CPAN and reduce the error in the Y data by a factor of 10, the covariance matrix does not change. Therefore, the covariance matrix is being normalized, but to what? Although it is likely normalized versus the sum of squared residuals, the implementation does not say. Further, the algorithm does not return any goodness of fit ( no SSR or chi squared).

Examination of the levmar documentation (according to the Wikipedia, PDL::Fit::LM is based on levmar) shows an array info that gives a much richer description of the fitting process, and access to the sum of squares required to assess goodness of fit. So I guess I'm still looking.