Edgeworth series

The Gram–Charlier A series and the Edgeworth series, named in honor of Francis Ysidro Edgeworth, are series that approximate a probability distribution in terms of its cumulants. The series are the same; but, the arrangement of terms (and thus the accuracy of truncating the series) differ.

Gram–Charlier A series

The key idea of these expansions is to write the characteristic function of the distribution whose probability density function is F to be approximated in terms of the characteristic function of a distribution with known and suitable properties, and to recover F through the inverse Fourier transform.

Let f be the characteristic function of the distribution whose density function is F, and κ_r its cumulants. We expand in terms of a known distribution with probability density function $\Psi$, characteristic function $\psi$, and standardized cumulants γ_r. The density $\Psi$ is generally chosen to be that of the normal distribution, but other choices are possible as well. By the definition of the cumulants, we have the following formal identity:

$f(t)=\exp \left[\sum _{r=1}^{\infty }(\kappa _{r}-\gamma _{r}){\frac {(it)^{r}}{r!}}\right]\psi (t)\,.$

By the properties of the Fourier transform, (it)^rψ(t) is the Fourier transform of (−1)^r D^r $\Psi$(x), where D is the differential operator with respect to x. Thus, we find for F the formal expansion

$F(x)=\exp \left[\sum _{r=1}^{\infty }(\kappa _{r}-\gamma _{r}){\frac {(-D)^{r}}{r!}}\right]\Psi (x)\,.$

If $\Psi$ is chosen as the normal density with mean and variance as given by F, that is, mean μ = κ₁ and variance σ² = κ₂, then the expansion becomes

$F(x)=\exp \left[\sum _{r=3}^{\infty }\kappa _{r}{\frac {(-D)^{r}}{r!}}\right]{\frac {1}{{\sqrt {2\pi }}\sigma }}\exp \left[-{\frac {(x-\mu )^{2}}{2\sigma ^{2}}}\right]\,.$

By expanding the exponential and collecting terms according to the order of the derivatives, we arrive at the Gram–Charlier A series. If we include only the first two correction terms to the normal distribution, we obtain

$F(x)={\frac {1}{{\sqrt {2\pi }}\sigma }}\exp \left[-{\frac {(x-\mu )^{2}}{2\sigma ^{2}}}\right]\left[1+{\frac {\kappa _{3}}{3!\sigma ^{3}}}H_{3}\left({\frac {x-\mu }{\sigma }}\right)+{\frac {\kappa _{4}}{4!\sigma ^{4}}}H_{4}\left({\frac {x-\mu }{\sigma }}\right)\right]\,,$

with H₃(x) = x³ − 3x and H₄(x) = x⁴ − 6x² + 3 (these are Hermite polynomials).

Note that this expression is not guaranteed to be positive, and is therefore not a valid probability distribution. The Gram–Charlier A series diverges in many cases of interest—it converges only if F(x) falls off faster than exp(−x²/4) at infinity (Cramér 1957). When it does not converge, the series is also not a true asymptotic expansion, because it is not possible to estimate the error of the expansion. For this reason, the Edgeworth series (see next section) is generally preferred over the Gram–Charlier A series.

Edgeworth series

Edgeworth developed a similar expansion as an improvement to the central limit theorem. The advantage of the Edgeworth series is that the error is controlled, so that it is a true asymptotic expansion.

Let {X_i} be a sequence of independent and identically distributed random variables with means μ and variances σ², and let Y_n be their standardized sums:

$Y_{n}={\frac {1}{\sqrt {n}}}\sum _{i=1}^{n}{\frac {X_{i}-\mu }{\sigma }}.$

Denote F_n the cumulative distribution functions of the variables Y_n. Then by the central limit theorem,

$\lim _{n\to \infty }F_{n}(x)=\Phi (x)\equiv \int _{-\infty }^{x}{\tfrac {1}{\sqrt {2\pi }}}e^{-{\frac {1}{2}}x^{2}}dx$

for every x, as long as the means and variances are finite and the sum of variances diverges to infinity.

Now assume that the random variables X_i have mean μ, variance σ², and higher cumulants κ_r=σ^rλ_r. If we expand in terms of the unit normal distribution, that is, if we set

$\Psi (x)={\frac {1}{\sqrt {2\pi }}}\exp(-{\tfrac {1}{2}}x^{2})$

then the cumulant differences in the formal expression of the characteristic function f_n(t) of F_n are

$\kappa _{1}^{F(n)}-\gamma _{1}=0\,,$

$\kappa _{2}^{F(n)}-\gamma _{2}=0\,,$

$\kappa _{r}^{F(n)}-\gamma _{r}={\frac {\kappa _{r}}{\sigma ^{r}n^{r/2-1}}}={\frac {\lambda _{r}}{n^{r/2-1}}};\quad r\geq 3\,.$

The Edgeworth series is developed similarly to the Gram–Charlier A series, only that now terms are collected according to powers of n. Thus, we have

$f_{n}(t)=\left[1+\sum _{j=1}^{\infty }{\frac {P_{j}(it)}{n^{j/2}}}\right]\exp(-t^{2}/2)\,,$

where P_j(x) is a polynomial of degree 3j. Again, after inverse Fourier transform, the density function F_n follows as

$F_{n}(x)=\Phi (x)+\sum _{j=1}^{\infty }{\frac {P_{j}(-D)}{n^{j/2}}}\Phi (x)\,.$

The first five terms of the expansion are

${\begin{aligned}F_{n}(x)=\ &\Phi (x)\\&-{\frac {1}{n^{1/2}}}{\bigg (}{\tfrac {1}{6}}\lambda _{3}\,\Phi ^{(3)}(x){\bigg )}\\&+{\frac {1}{n}}{\bigg (}{\tfrac {1}{24}}\lambda _{4}\,\Phi ^{(4)}(x)+{\tfrac {1}{72}}\lambda _{3}^{2}\,\Phi ^{(6)}(x){\bigg )}\\&-{\frac {1}{n^{3/2}}}{\bigg (}{\tfrac {1}{120}}\lambda _{5}\,\Phi ^{(5)}(x)+{\tfrac {1}{144}}\lambda _{3}\lambda _{4}\,\Phi ^{(7)}(x)+{\tfrac {1}{1296}}\lambda _{3}^{3}\,\Phi ^{(9)}(x){\bigg )}\\&+{\frac {1}{n^{2}}}{\bigg (}{\tfrac {1}{720}}\lambda _{6}\,\Phi ^{(6)}(x)+{\big (}{\tfrac {1}{1152}}\lambda _{4}^{2}+{\tfrac {1}{720}}\lambda _{3}\lambda _{5}{\big )}\Phi ^{(8)}(x)\\&\qquad \quad +{\tfrac {1}{1728}}\lambda _{3}^{2}\lambda _{4}\,\Phi ^{(10)}(x)+{\tfrac {1}{31104}}\lambda _{3}^{4}\,\Phi ^{(12)}(x){\bigg )}\\&+O(n^{-5/2})\,.\end{aligned}}$

Here, Φ^(j)(x) is the j-th derivative of Φ(·) at point x. Blinnikov and Moessner (1998) have given a simple algorithm to calculate higher-order terms of the expansion.

Further reading

• H. Cramér (1957). Mathematical Methods of Statistics. Princeton University Press, Princeton.
• D. L. Wallace (1958). "Asymptotic approximations to distributions". Annals of Mathematical Statistics 29:635–654.
• M. Kendall & A. Stuart (1977), The advanced theory of statistics, Vol 1: Distribution theory, 4th Edition, Macmillan, New York
• P. McCullagh (1987). Tensor Methods in Statistics. Chapman and Hall, London.
• D. R. Cox and O. E. Barndorff-Nielsen (1989). Asymptotic Techniques for Use in Statistics. Chapman and Hall, London.
• P. Hall (1992). The Bootstrap and Edgeworth Expansion. Springer, New York.
• S. Blinnikov and R. Moessner (1998). Expansions for nearly Gaussian distributions. Astron. Astrophys. Suppl. Ser. 130:193–205.
• J. E. Kolassa (2006). Series Approximation Methods in Statistics, 3rd Edition. (Lecture Notes in Statistics #88). Springer, New York.