Heteroscedasticity

From Wikipedia, the free encyclopedia

This article only describes one highly specialized aspect of its associated subject. Please help improve this article by adding more general information. (October 2009)

In statistics, a sequence of random variables is heteroscedastic, or heteroskedastic, if the random variables have different variances. The term means "differing variance" and comes from the Greek "hetero" ('different') and "skedasis" ('dispersion'). In contrast, a sequence of random variables is called homoscedastic if it has constant variance.

Suppose there is a sequence of random variables {Y_t}_t=1ⁿ and a sequence of vectors of random variables, {X_t}_t=1ⁿ. In dealing with conditional expectations of Y_t given X_t, the sequence {Y_t}_t=1ⁿ is said to be heteroscedastic if the conditional variance of Y_t given X_t, changes with t. Some authors refer to this as conditional heteroscedasticity to emphasize the fact that it is the sequence of conditional variance that changes and not the unconditional variance. In fact it is possible to observe conditional heteroscedasticity even when dealing with a sequence of unconditional homoscedastic random variables, however, the opposite does not hold.

When using some statistical techniques, such as ordinary least squares (OLS), a number of assumptions are typically made. One of these is that the error term has a constant variance. This will be true if the observations of the error term are assumed to be drawn from identical distributions. Heteroscedasticity may in some instances violate this assumption.

For example, the error term could vary or increase with each observation, something that is often the case with cross-sectional or time series measurements. Heteroscedasticity is often studied as part of econometrics, which frequently deals with data exhibiting it.

With the advent of robust standard errors allowing for inference without specifying the conditional second moment of error term, testing conditional homoscedasticity is not as important as in the past.^{[citation needed]}

The econometrician Robert Engle won the 2003 Nobel Memorial Prize for Economics for his studies on regression analysis in the presence of heteroscedasticity, which led to his formulation of the Autoregressive conditional heteroskedasticity (ARCH) modeling technique.

Contents 1 Consequences 2 Detection 3 Fixes 4 Examples 5 See also 6 References 7 Further reading

[edit] Consequences

Heteroscedasticity does not cause OLS coefficient estimates to be biased nor inconsistent, but it can cause the variance (and, thus, standard errors) of the coefficients to be underestimated. Thus, a regression analysis using heteroscedastic data will still provide the best estimate for the relationship between the predictor variable and the outcome, but it may judge the relationship to be statistically significant when it is actually too weak to be confidently distinguished from zero.

It is widely known that, under certain conditions, the OLS estimator has a normal asymptotic distribution when properly normalized and centered (even when the data does not come from a normal distribution). This result is used to justify using a normal distribution, or a chi square distribution (depending on how the test statistic is calculated), when doing hypothesis test. This holds even under heteroscedasticity. More precisely, the OLS estimator in the presence of heteroscedasticity, is asymptotically normal, when properly normalized and centered, with a variance-covariance matrix that differs from the case of homoscedasticity. White (1980) ^[1], proposed a consistent estimator for the variance-covariance matrix of the asymptotic distribution of the OLS estimator. This validates the use of hypothesis testing using OLS estimators and White's variance-covariance estimator under heteroscedasticity.

Heteroscedasticity is also a major practical issue encountered in ANOVA problems.^[2] The F test can still be used in some circumstances.^[3]

[edit] Detection

There are several methods to test for the presence of heteroscedasticity:

• Park test (1966)^[4]
• Glejser test (1969)
• White test (1980)
• Breusch–Pagan test
• Goldfeld–Quandt test
• Cook–Weisberg test
• Harrison–McCabe test
• Brown–Forsythe test
• Levene test

These methods consists, in general, in performing hypothesis tests. These tests consists a statistic (a mathematical expression), a hypothesis that is going to be tested (null hypothesis), an alternative hypothesis and a distributional statement about the statistic (the mathematical expression).

Many introductory statistics and econometrics books, for pedagogical reasons, presents these tests under the assumption that the data set in hand comes from a normal distribution. A great misconception is the thought that this assumption is necessary, however, in most cases this assumption can be relaxed by using asymptotic distribution theory (aka asymptotic theory).

Most of the methods of detecting heteroscedasticity presented here can be used even when the data does not come from a normal distribution. This is done by realizing that the distributional statement about the statistic (the mathematical expression) can be approximated by using asymptotic theory.

[edit] Fixes

There are three common corrections for heteroscedasticity:

• Use a different specification for the model (different X variables, or perhaps non-linear transformations of the X variables).
• Apply a weighted least squares estimation method, in which OLS is applied to transformed or weighted values of X and Y. The weights vary over observations, depending on the changing error variances.
• Heteroscedasticity-consistent standard errors (HCSE), while still biased, improve upon OLS estimates (White 1980). Generally, HCSEs are greater than their OLS counterparts, resulting in lower t-scores and a reduced probability of statistically significant coefficients. The White method corrects for heteroscedasticity without altering the values of the coefficients. This method may be superior to regular OLS because if heteroscedasticity is present it corrects for it, however, if it is not present you have not made any error.

[edit] Examples

Heteroscedasticity often occurs when there is a large difference among the sizes of the observations.

• A classic example of heteroscedasticity is that of income versus expenditure on meals. As one's income increases, the variability of food consumption will increase. A poorer person will spend a rather constant amount by always eating fast food; a wealthier person may occasionally buy fast food and other times eat an expensive meal. Those with higher incomes display a greater variability of food consumption.
• Imagine you are watching a rocket take off nearby and measuring the distance it has traveled once each second. In the first couple of seconds your measurements may be accurate to the nearest centimeter, say. However, 5 minutes later as the rocket recedes into space, the accuracy of your measurements may only be good to 100 m, because of the increased distance, atmospheric distortion and a variety of other factors. The data you collect would exhibit heteroscedasticity.

[edit] See also

• Kurtosis (peakedness)
• Breusch–Pagan test of heteroscedasticity of the residuals of a linear regression
• Regression analysis
• homoscedasticity
• Autoregressive conditional heteroscedasticity (ARCH)
• White test
• Heteroscedasticity-consistent standard errors

[edit] References

[edit] Further reading

Most statistics textbooks will include at least some material on heteroscedasticity. Some examples are:

• Studenmund AH. Using Econometrics (2nd ed.). ISBN 0-673-52125-7.  (devotes a chapter to heteroscedasticity)
• Verbeek, Marno (2004): A Guide to Modern Econometrics, 2. ed., Chichester: John Wiley & Sons, 2004, pages
• Greene, W.H. (1993), Econometric Analysis, Prentice–Hall, ISBN 0-13-013297-7, an introductory but thorough general text, considered the standard for a pre-doctorate university Econometrics course;
• Hamilton, J.D. (1994), Time Series Analysis, Princeton University Press ISBN 0-691-04289-6, the text of reference for historical series analysis; it contains an introduction to ARCH models.
• Vinod, H.D. (2008): Hands On Intermediate Econometrics Using R: Templates for Extending Dozens of Practical Examples. ISBN 10-981-281-885-5 (Section 2.8 provides R snippets) World Scientific Publishers: Hackensack, NJ .
• White, Halbert (1980). "A heteroscedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroscedasticity". Econometrica 48: 817–838. 

Special subjects

• Glejser test: Furno, Marilena (2005). "The Glejser Test and the Median Regression". Sankhya – The Indian Journal of Statistics, Special Issue on Quantile Regression and Related Methods 67 (2): 335–358. http://sankhya.isical.ac.in/search/67_2/2005015.pdf.  (work done at Universita di Cassino, Italy, 2005)
• heteroscedasticity in QSAR Modeling

v • d • e Statistics Descriptive statistics Continuous data Location Mean (Arithmetic, Geometric, Harmonic) · Median · Mode Dispersion Range · Standard deviation · Coefficient of variation · Percentile Moments Variance · Semivariance · Skewness · Kurtosis Categorical data Frequency · Contingency table Inferential statistics and hypothesis testing Inference Confidence interval (Frequentist inference) · Credible interval (Bayesian inference) · Significance · Meta-analysis Design of experiments Population · Sampling · Stratified sampling · Replication · Blocking · Sensitivity and specificity · Optimal design Sample size estimation Statistical power · Effect size · Standard error General estimation Bayesian estimator · Maximum likelihood · Method of moments · Minimum distance · Maximum spacing Specific tests Z-test (normal) · Student's t-test · F-test · Chi-square test · Pearson's chi-square test · Wald test · Mann–Whitney U · Wilcoxon signed-rank test Survival analysis Survival function · Kaplan–Meier · Logrank test · Failure rate · Proportional hazards models Correlation and regression analysis Correlation Pearson product-moment correlation · Rank correlation (Spearman's rho, Kendall's tau) · Confounding variable Linear regression Simple linear regression · Ordinary least squares · General linear model · Analysis of variance · Analysis of covariance Non-standard Nonlinear regression · Nonparametric · Semiparametric · Robust Non-normal errors Generalized linear model · Binomial · Poisson · Logistic Statistical graphics Bar chart · Biplot · Box plot · Control chart · Correlogram · Forest plot · Histogram · Q-Q plot · Run chart · Scatter plot · Stemplot Category · Portal · Outline · Index