Goodness of fit

From Wikipedia, the free encyclopedia

The goodness of fit of a statistical model describes how well it fits a set of observations. Measures of goodness of fit typically summarize the discrepancy between observed values and the values expected under the model in question. Such measures can be used in statistical hypothesis testing, e.g. to test for normality of residuals, to test whether two samples are drawn from identical distributions (see Kolmogorov-Smirnov test), or whether outcome frequencies follow a specified distribution (see Pearson's chi-square test). In the analysis of variance, one of the components into which the variance is partitioned may be a lack-of-fit sum of squares.

Contents 1 Example 2 Binomial case 3 References 4 See also

[edit] Example

The chi-square statistic is a sum of differences between observed and expected outcome frequencies, each squared and divided by the expectation:

where:

O = an observed frequency

E = an expected (theoretical) frequency, asserted by the null hypothesis

The resulting value can be compared to the chi-square distribution to determine the goodness of fit.

In order to determine the degrees of Freedom of the Chi-Squared distribution, one takes the total number of observed frequencies and subtracts one. For example, if there are eight different frequencies, one would compare to a chi-squared with seven degrees of freedom.

Another way to describe the chi-squared statistic is with the differences weighted based on measurement error:

where σ² is the variance of the observation.^[1] This definition is useful when one has estimates for the error on the measurements.

The reduced chi-squared statistic is simply the chi-squared divided by the number of degrees of freedom: ^{[1] [2] [3] [4]}

where ν is the number of degrees of freedom, usually given by N − n, where N is the number of data points, and n is the number of fit parameters. The advantage of the reduced chi-squared is that it already normalizes for the number of data points and model complexity. As a rule of thumb, a large indicates a poor model fit. However indicates that the model is 'over-fitting' the data (either the model is improperly fitting noise, or the error bars have been over-estimated). A indicates that the fit has not fully captured the data (or that the error bars have been under-estimated). In principle a is the best-fit for the given data and error bars.

[edit] Binomial case

A binomial experiment is a sequence of independent trials in which the trials can result in one of two outcomes, success or failure. There are n trials each with probability of success, denoted by p. Provided that np_i ≫ 1 for every i (where i = 1, 2, ..., k), then

This has approximately a chi-squared distribution with k − 1 df. The fact that df = k − 1 is a consequence of the restriction . We know there are k observed cell counts, however, once any k − 1 are known, the remaining one is uniquely determined. Basically, one can say, there are only k − 1 freely determined cell counts, thus df = k − 1.

[edit] References

1. ^ ^{a b} Charlie Laub and Tonya L. Kuhl: Chi-Square Data Fitting. University California, Davis.
2. ^ John Robert Taylor: An introduction to error analysis, page 268. University Science Books, 1997.
3. ^ Kirkman, T.W.: Chi-Square Curve Fitting.
4. ^ David M. Glover, William J. Jenkins, and Scott C. Doney: Least Squares and regression techniques, goodness of fit and tests, non-linear least squares techniques. Woods Hole Oceanographic Institute, 2008.

[edit] See also

• Coefficient of determination (R squared)