Asymptotic theory (statistics)

In statistics, asymptotic theory, or large sample theory, is a framework for assessing properties of estimators and statistical tests. Within this framework, it is often assumed that the sample size n may grow indefinitely; the properties of estimators and tests are then evaluated under the limit of n → ∞. In practice, a limit evaluation is considered to be approximately valid for large finite sample sizes too.

Overview

Most statistical problems begin with a dataset of size n. The asymptotic theory proceeds by assuming that it is possible (in principle) to keep collecting additional data, thus that the sample size grows infinitely, i.e. n → ∞. Under the assumption, many results can be obtained that are unavailable for samples of finite size. An example is the weak law of large numbers. The law states that for a sequence of independent and identically distributed (IID) random variables X₁, X₂, ..., if one value is drawn from each random variable and the average of the first n values is computed as X_n, then the X_n converge in probability to the population mean E[X_i] as n → ∞.

In asymptotic theory, the standard approach is n → ∞. For some statistical models, slightly different approaches of asymptotics may be used. For example, with panel data, it is commonly assumed that one dimension in the data remains fixed, whereas the other dimension grows: T = constant and N → ∞, or vice versa.

Besides the standard approach to asymptotics, other alternative approaches exist:

• Within the local asymptotic normality framework, it is assumed that the value of the "true parameter" in the model varies slightly with n, such that the n-th model corresponds to θ_n = θ + h/√n . This approach lets us study the regularity of estimators.
• When statistical tests are studied for their power to distinguish against the alternatives that are close to the null hypothesis, it is done within the so-called "local alternatives" framework: the null hypothesis is H₀: θ = θ₀ and the alternative is H₁: θ = θ₀ + h/√n . This approach is especially popular for the unit root tests.
• There are models where the dimension of the parameter space Θ_n slowly expands with n, reflecting the fact that the more observations there are, the more structural effects can be feasibly incorporated in the model.
• In kernel density estimation and kernel regression, an additional parameter is assumed—the bandwidth h. In those models, it is typically taken that h → 0 as n → ∞. The rate of convergence must be chosen carefully, though, usually h ∝ n^−1/5.

In many cases, highly accurate results for finite samples can be obtained via numerical methods (i.e. computers); even in such cases, though, asymptotic analysis can be useful. This point was made by Small (2010, §1.4), as follows.

A primary goal of asymptotic analysis is to obtain a deeper qualitative understanding of quantitative tools. The conclusions of an asymptotic analysis often supplement the conclusions which can be obtained by numerical methods.

Modes of convergence of random variables

Further information: Convergence of random variables

Asymptotic properties

Point estimators

In the following, $({\hat {\theta }}_{n})_{n\in \mathbb {N} }$ will be an arbitrary sequence of estimators.

Consistency

The sequence $({\hat {\theta }}_{n})_{n\in \mathbb {N} }$ is said to be (weakly) consistent, if it converges in probability to the true value of the parameter being estimated:

${\hat {\theta }}_{n}\ {\xrightarrow {\overset {}{p}}}\ \theta _{0}.$

This means that the probability that the estimated value gets arbitrarily close to $\theta _{0}$ converges to zero as $n$ tends to infinity. The sequence of estimates is said be strongly consistent, if it even converges almost surely to the true parameter:$${\hat {\theta }}_{n}{\overset {\mathrm {a.s.} }{\longrightarrow }}\ \theta _{0}.$$The subtle difference to a weakly consistent estimator is that a strongly consistent estimator is guaranteed to get arbitrarily close to $\theta _{0}$ as $n$ tends to infinity, eventually.

Asymptotic distribution

If it is possible to find sequences of non-random constants $(a_{n})_{n\in \mathbb {N} }$, $(b_{n})_{n\in \mathbb {N} }$ (possibly depending on the value $\theta _{0}$), and a non-degenerate distribution $G$ such that

$b_{n}({\hat {\theta }}_{n}-a_{n})\ {\xrightarrow {d}}\ G,$

then the sequence of estimators ${\hat {\theta }}_{n}$ is said to have the asymptotic distribution $G$.

Most often, the estimators encountered in practice are asymptotically normal, meaning their asymptotic distribution is the normal distribution, with $a_{n}=\theta _{0}$, $b_{n}={\sqrt {n}}$, and $G={\mathcal {N}}(0,V)$ for some covariance matrix $V$:

${\sqrt {n}}({\hat {\theta }}_{n}-\theta _{0})\ {\xrightarrow {d}}\ {\mathcal {N}}(0,V).$

Asymptotic unbiasedness

The sequence is called asymptotically unbiased for the parameter $\theta _{0}$, if $\sigma _{n}({\hat {\theta }}_{n}-\theta _{0})$ converges in distribution to a random variable $G\neq 0$ with $\mathbb {E} [G]=0$ for some sequence $(\sigma _{n})_{n\in \mathbb {N} }$ of real numbers.

Limiting variance

If there exists a sequence $(\tau _{n})_{n\in \mathbb {N} }$ of real numbers such that $\lim _{n\to \infty }\tau _{n}\mathrm {Var} ({{\hat {\theta }}_{n}})=\sigma ^{2}$, then $\sigma ^{2}$ is called the limiting variance of $({\hat {\theta }}_{n})_{n\in \mathbb {N} }$. Typically, it is chosen $\tau _{n}=n$.

Asymptotic variance

If the sequence is asymptotically normal for $\theta _{0}$, that is, ${\sqrt {n}}({\hat {\theta }}_{n}-\theta _{0})\xrightarrow {d} {\mathcal {N}}(0,V)$, then $V$ is called the asymptotic variance of $({\hat {\theta }}_{n})_{n\in \mathbb {N} }$.

In the typical (univariate) case, the asymptotic variance will coincide with the limiting variance. In general, however, the two concepts might differ and, if $\sigma ^{2}<\infty$, it always holds $\sigma ^{2}\leq V$.

Asymptotic efficiency

The sequence is called asymptotically efficient, if is asymptotically normal for $\theta _{0}$ and its asymptotic variance is $V=I(\theta _{0})^{-1}$ with the Fisher information $I(\theta _{0})$, that is, the asymptotic variance achieves the Cramér–Rao bound.

Asymptotic theorems

Asymptotic properties of a sequence of estimators $({\hat {\theta }}_{n})_{n\in \mathbb {N} }$ (for a parameter $\theta$) often result from general theorems from probability theory. In the following, we let $X_{1},,X_{2},\dots$ be independent and identically distributed observations from the true distribution $\mathbb {P} _{\theta }$.

For consistency

Strong law of large numbers

If the estimators are of the form $\theta _{n}={\frac {1}{n}}\sum _{i=1}^{n}f(X_{i})$ for some measurable function $f$ such that $\mathbb {E} [|f(X_{1})|]<\infty$, then by the strong law of large numbers

$${\hat {\theta }}_{n}\xrightarrow {a.s.} \mathbb {E} [f(X_{1})].$$Thus, if $\theta =\mathbb {E} [f(X_{1})]$, then $({\hat {\theta }}_{n})_{n\in \mathbb {N} }$ is strongly consistent.

Continuous mapping theorem

If the estimators are of the form $\theta _{n}=f(\tau _{n})$ for some continuous function $f$ and a sequence of estimators $\tau _{n}\xrightarrow {a.s.} \tau$, then by the continuous mapping theorem $\theta _{n}\xrightarrow {a.s.} f(\tau )$. The same holds true if almost sure convergence is replaced by convergence in probability.

Thus, if $\theta =f(\tau )$, then $({\hat {\theta }}_{n})_{n\in \mathbb {N} }$ is (strongly) consistent.

For asymptotic distribution

• Central limit theorem
• Fisher–Tippet–Gnedenko theorem
• Glivenko–Cantelli theorem
• Slutsky's theorem
• Delta method

See also

• Asymptotic analysis
• Exact statistics
• Large deviations theory

Bibliography

• Balakrishnan, N.; Ibragimov, I. A. V. B.; Nevzorov, V. B., eds. (2001), Asymptotic Methods in Probability and Statistics with Applications, Birkhäuser, ISBN 9781461202097
• Borovkov, A. A.; Borovkov, K. A. (2010), Asymptotic Analysis of Random Walks, Cambridge University Press
• Buldygin, V. V.; Solntsev, S. (1997), Asymptotic Behaviour of Linearly Transformed Sums of Random Variables, Springer, ISBN 9789401155687
• Le Cam, Lucien; Yang, Grace Lo (2000), Asymptotics in Statistics (2nd ed.), Springer
• Casella, G.; Berger, R.L. (2002), Statistical Inference (2nd ed.), Duxbury
• Dawson, D.; Kulik, R.; Ould Haye, M.; Szyszkowicz, B.; Zhao, Y., eds. (2015), Asymptotic Laws and Methods in Stochastics, Springer-Verlag
• Höpfner, R. (2014), Asymptotic Statistics, Walter de Gruyter
• Lehmann, E.L.; Casella, G. (1998), Theory of Point Estimation (2nd ed.), Springer
• Lin'kov, Yu. N. (2001), Asymptotic Statistical Methods for Stochastic Processes, American Mathematical Society
• Oliveira, P. E. (2012), Asymptotics for Associated Random Variables, Springer
• Petrov, V. V. (1995), Limit Theorems of Probability Theory, Oxford University Press
• Sen, P. K.; Singer, J. M.; Pedroso de Lima, A. C. (2009), From Finite Sample to Asymptotic Methods in Statistics, Cambridge University Press
• Shiryaev, A. N.; Spokoiny, V. G. (2000), Statistical Experiments and Decisions: Asymptotic theory, World Scientific
• Small, C. G. (2010), Expansions and Asymptotics for Statistics, Chapman & Hall
• van der Vaart, A. W. (1998), Asymptotic Statistics, Cambridge University Press