Berry–Esseen theorem

The central limit theorem in probability theory and statistics states that under certain circumstances the sample mean, considered as a random quantity, becomes more normally distributed as the sample size is increased. The Berry–Esseen theorem, also known as the Berry–Esseen inequality, attempts to quantify the rate at which this convergence to normality takes place.

Statement of the theorem

Statements of the theorem vary, as it was independently discovered by two mathematicians, Andrew C. Berry (in 1941) and Carl-Gustav Esseen (1942), who then, along with other authors, refined it repeatedly over subsequent decades.

Identically distributed summands

One version, sacrificing generality somewhat for the sake of clarity, is the following:

Let X₁, X₂, ..., be i.i.d. random variables with E(X₁) = 0, E(X₁²) = σ² > 0, and E(|X₁|³) = ρ < ∞. Also, let

$Y_{n}={X_{1}+X_{2}+\cdots +X_{n} \over n}$

be the sample mean, with F_n the cdf of

${Y_{n}{\sqrt {n}} \over {\sigma }},$

and Φ the cdf of the standard normal distribution. Then there exists a positive constant C such that for all x and n,

$\left|F_{n}(x)-\Phi (x)\right|\leq {C\rho \over \sigma ^{3}\,{\sqrt {n}}}.\ \ \ \ (1)$

Illustration of the difference in cumulative distribution functions alluded to in the theorem.

That is: given a sequence of independent and identically-distributed random variables, each having mean zero and positive variance, if additionally the third absolute moment is finite, then the cumulative distribution functions of the standardized sample mean and the standard normal distribution differ (vertically, on a graph) by no more than the specified amount. Note that the rate of convergence is on the order of n^−1/2.

Calculated values of the constant C have decreased markedly over the years, from the original value of 7.59 by Esseen (1942), to 0.7882 by van Beek (1972), then 0.7655 by Shiganov (1986), then 0.7056 by Shevtsova (2007), then 0.7005 by Shevtsova (2008), then 0.5894 by Tyurin (2009), then 0.5129 by Korolev & Shevtsova (2009), then 0.4785 by Tyurin (2010). The detailed review can be found in the papers Korolev & Shevtsova (2009), Korolev & Shevtsova (2010). The best estimate as of 2011, C<0.4784, follows from the inequality

$\sup _{x}\left|F_{n}(x)-\Phi (x)\right|\leq {0.33477(\rho +0.429\sigma ^{3}) \over \sigma ^{3}\,{\sqrt {n}}},$

due to Korolev & Shevtsova (2010), since σ³≤ρ and 0.33477·1.429<0.4784.

Esseen (1956) proved that the bound must satisfy

$C\geq {\frac {{\sqrt {10}}+3}{6{\sqrt {2\pi }}}}\approx 0.40973\approx {\frac {1}{\sqrt {2\pi }}}+0.01079.$

Non-identically distributed summands

Let X₁, X₂, ..., be independent random variables with E(X_i) = 0, E(X_i²) = σ_i² > 0, and E(|X_i|³) = ρ_i < ∞. Also, let

$S_{n}={X_{1}+X_{2}+\cdots +X_{n} \over {\sqrt {\sigma _{1}^{2}+\sigma _{2}^{2}+\cdots +\sigma _{n}^{2}}}}$

be the normalized n-th partial sum. Denote F_n the cdf of S_n, and Φ the cdf of the standard normal distribution. For the sake of convenience denote

${\vec {\sigma }}=(\sigma _{1},...,\sigma _{n}),\ {\vec {\rho }}=(\rho _{1},...,\rho _{n}).$

In 1941, Andrew C. Berry proved that for all n there exists an absolute constant C₁ such that

$\sup _{x}\left|F_{n}(x)-\Phi (x)\right|\leq C_{1}\cdot \psi _{1},\ \ \ \ (2)$

where

$\psi _{1}=\psi _{1}{\big (}{\vec {\sigma }},{\vec {\rho }}{\big )}={\Big (}{\textstyle \sum \limits _{i=1}^{n}\sigma _{i}^{2}}{\Big )}^{-1/2}\cdot \max _{1\leq i\leq n}{\frac {\rho _{i}}{\sigma _{i}^{2}}}.$

Independently, in 1942, Carl-Gustav Esseen proved that for all n there exists an absolute constant C₀ such that

$\sup _{x}\left|F_{n}(x)-\Phi (x)\right|\leq C_{0}\cdot \psi _{0},\ \ \ \ (3)$

where

$\psi _{0}=\psi _{0}{\big (}{\vec {\sigma }},{\vec {\rho }}{\big )}={\Big (}{\textstyle \sum \limits _{i=1}^{n}\sigma _{i}^{2}}{\Big )}^{-3/2}\cdot \sum \limits _{i=1}^{n}\rho _{i}.$

It is easy to make sure that ψ₀≤ψ₁. Due to this circumstance inequality (3) is conventionally called the Berry-Esseen inequality, and the quantity ψ₀ is called the Lyapunov fraction of the third order. Moreover, in the case where the summands X₁,... X_n have identical distributions

$\psi _{0}=\psi _{1}={\frac {\rho _{1}}{\sigma _{1}^{3}{\sqrt {n}}}},$

and thus the bounds stated by inequalities (1), (2) and (3) coincide.

Regarding C₀, obviously, the lower bound established by Esseen (1956) remains valid:

$C_{0}\geq {\frac {{\sqrt {10}}+3}{6{\sqrt {2\pi }}}}=0.4097\ldots .$

The upper bounds for C₀ were subsequently lowered from the original estimate 7.59 due to Esseen (1942) to (we mention the recent results only) 0.9051 due to Zolotarev (1967), 0.7975 due to van Beek (1972), 0.7915 due to Shiganov (1986), 0.6379 and 0.5606 due to Tyurin (2009) and Tyurin (2010). As of 2011 the best estimate is 0.5600 obtained by Shevtsova (2010).

See also

• Chernoff's inequality
• Edgeworth series
• List of inequalities
• List of mathematical theorems

References

• Berry, Andrew C. (1941). "The Accuracy of the Gaussian Approximation to the Sum of Independent Variates". Transactions of the American Mathematical Society. 49 (1): 122–136. doi:10.1090/S0002-9947-1941-0003498-3. JSTOR 1990053. {{cite journal}}: Invalid |ref=harv (help)
• Durrett, Richard (1991). Probability: Theory and Examples. Pacific Grove, CA: Wadsworth & Brooks/Cole. ISBN 0-534-13206-5.
• Esseen, Carl-Gustav (1942). "On the Liapunoff limit of error in the theory of probability". Arkiv för matematik, astronomi och fysik. A28: 1–19. ISSN 0365-4133. {{cite journal}}: Invalid |ref=harv (help)
• Esseen, Carl-Gustav (1956). "A moment inequality with an application to the central limit theorem". Skand. Aktuarietidskr. 39: 160–170. {{cite journal}}: Invalid |ref=harv (help)
• Feller, William (1972). An Introduction to Probability Theory and Its Applications, Volume II (2nd ed.). New York: John Wiley & Sons. ISBN 0-471-25709-5.
• Korolev, V. Yu.; Shevtsova, I. G. (2010). "On the upper bound for the absolute constant in the Berry-Esseen inequality". Theory of Probability and its Applications. 54 (4): 638–658. doi:10.1137/S0040585X97984449. {{cite journal}}: Invalid |ref=harv (help)
• Korolev, Victor; Shevtsova, Irina (2010). "An improvement of the Berry-Esseen inequality with applications to Poisson and mixed Poisson random sums". Scandinavian Actuarial Journal: 1–25. doi:10.1080/03461238.2010.485370. {{cite journal}}: Invalid |ref=harv (help)
• Manoukian, Edward B. (1986). Modern Concepts and Theorems of Mathematical Statistics. New York: Springer-Verlag. ISBN 0-387-96186-0.
• Serfling, Robert J. (1980). Approximation Theorems of Mathematical Statistics. New York: John Wiley & Sons. ISBN 0-471-02403-1.
• Shevtsova, I. G. (2008). "On the absolute constant in the Berry-Esseen inequality". The Collection of Papers of Young Scientists of the Faculty of Computational Mathematics and Cybernetics (5): 101–110. {{cite journal}}: Invalid |ref=harv (help)
• Shevtsova, I. G. (2007). "Sharpening of the upper bound of the absolute constant in the Berry–Esseen inequality". Theory of Probability and its Applications. 51 (3): 549–553. doi:10.1137/S0040585X97982591. {{cite journal}}: Invalid |ref=harv (help)
• Shevtsova, I. G. (2010). "An Improvement of Convergence Rate Estimates in the Lyapunov Theorem". Doklady Mathematics. 82 (3): 862–864. doi:10.1134/S1064562410060062. {{cite journal}}: Invalid |ref=harv (help)
• Shiganov, I.S. (1986). "Refinement of the upper bound of a constant in the remainder term of the central limit theorem". Journal of Soviet mathematics. 35 (3): 109–115. doi:10.1007/BF01121471. {{cite journal}}: Invalid |ref=harv (help)
• Tyurin, I.S. (2009). "On the accuracy of the Gaussian approximation". Doklady Mathematics. 80 (3): 840–843. doi:10.1134/S1064562409060155. {{cite journal}}: Invalid |ref=harv (help)
• Tyurin, I.S. (2010). "An improvement of upper estimates of the constants in the Lyapunov theorem". Russian Mathematical Surveys. 65 (3(393)): 201–202. {{cite journal}}: Invalid |ref=harv (help)
• van Beek, P. (1972). "An application of Fourier methods to the problem of sharpening the Berry–Esseen inequality". Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete. 23 (3): 187–196. doi:10.1007/BF00536558. {{cite journal}}: Invalid |ref=harv (help)
• Zolotarev, V. M. (1967). "A sharpening of the inequality of Berry–Esseen". Z. Wahrsch. Verw. Geb. 8 (4): 332–342. doi:10.1007/BF00531598. {{cite journal}}: Invalid |ref=harv (help)

External links

• Gut, Allan & Holst Lars. Carl-Gustav Esseen, retrieved Mar. 15, 2004.