Edgeworth series explained

The Gram–Charlier A series (named in honor of Jørgen Pedersen Gram and Carl Charlier), and the Edgeworth series (named in honor of Francis Ysidro Edgeworth) are series that approximate a probability distribution in terms of its cumulants.^[1] The series are the same; but, the arrangement of terms (and thus the accuracy of truncating the series) differ.^[2] The key idea of these expansions is to write the characteristic function of the distribution whose probability density function is to be approximated in terms of the characteristic function of a distribution with known and suitable properties, and to recover through the inverse Fourier transform.

Gram–Charlier A series

We examine a continuous random variable. Let

\hat{f} be the characteristic function of its distribution whose density function is, and \kappa _{r} its cumulants. We expand in terms of a known distribution with probability density function, characteristic function \hat{\psi} , and cumulants \gamma _{r} . The density is generally chosen to be that of the normal distribution, but other choices are possible as well. By the definition of the cumulants, we have (see Wallace, 1958)^[3] \hat{f}(t) = \exp\left[\sum ^{inf ty\kappa} _{r ((it) ^{r})/(r!)} \right] and \hat{\psi}(t) = \exp\left[\sum ^{inf ty\gamma} _{r ((it) ^{r})/(r!)} \right], which gives the following formal identity:\hat{f}(t) = \exp\left[\sum ^{inf ty(\kappa} _{r - \gamma} ^{r) ((it) ^{r})/(r!)} \right]\hat{\psi}(t).

By the properties of the Fourier transform,

(it) ^{r} \hat{\psi}(t) is the Fourier transform of ( -1 ) ^{r[D} ^{r\psi](-x)} , where is the differential operator with respect to . Thus, after changing x with - x on both sides of the equation, we find for the formal expansionf(x) = \exp\left[\sum ^{inf ty(\kappa} _{r} - \gamma _{r) (( - D) ^{r})/(r!)} \right]\psi(x).

If is chosen as the normal density

\phi(x) = (1)/(\sigma}\exp\left[ - \sqrt{ 2 \pi) ((x - \mu) ^{2})/(2 \sigma ^{2}) \right]

with mean and variance as given by, that is, mean

\mu = \kappa _{1} and variance \sigma ^{2} = \kappa _{2} , then the expansion becomesf(x) = \exp\left[\sum ^{inf ty\kappa} _{r (( - D) ^{r})/(r!)} \right] \phi(x),

since

\gamma _{r=0} for all > 2, as higher cumulants of the normal distribution are 0. By expanding the exponential and collecting terms according to the order of the derivatives, we arrive at the Gram–Charlier A series. Such an expansion can be written compactly in terms of Bell polynomials as\exp\left[\sum ^{inf ty\kappa} _{r (( - D) ^{r})/(r!)} \right] = \sum ^{inf ty} _{n =0} B _{n(0,0,\kappa} _{3,\ldots,\kappa} ^{n) (( - D) ^{n})/(n!)} .

Since the n-th derivative of the Gaussian function

\phi is given in terms of Hermite polynomial as\phi ^{(n)} (x) = (( -1 ) ^{n})/(\sigma ^{n}) He _{n} \left( (x - \mu)/(\sigma) \right) \phi(x),

this gives us the final expression of the Gram–Charlier A series as

f(x) = \phi(x) \sum ^{inf ty} _{n =0} (1)/(n! \sigma ^{n}) B _{n(0,0,\kappa} _{3,\ldots,\kappa} _{n)} He _{n} \left( (x - \mu)/(\sigma) \right). F(x) = \int ^{x} _{-inf ty} f(u) du = \Phi(x) - \phi(x) \sum ^{inf ty} _{n =3} (1)/(n! \sigma ^{n-1}) B _{n(0,0,\kappa} _{3,\ldots,\kappa} _{n)} He _{n-1} \left( (x - \mu)/(\sigma) \right),

where

\Phi is the CDF of the normal distribution.

If we include only the first two correction terms to the normal distribution, we obtain

f(x) ≈ (1)/(\sigma}\exp\left[ - \sqrt{ 2 \pi) ((x - \mu) ^{2})/(\right]\left[ 1+ 2 \sigma ^{2}) (\kappa _{3})/(3 !\sigma ^{3}) He _{3 \left( (x - \mu)/(\sigma)} \right) + (\kappa _{4})/(4 !\sigma ^{4}) He _{4 \left( (x - \mu)/(\sigma)} \right)\right],

with

He ^{3-3 x} _{3 (x) = x} and He ^{4} _{4 (x) = x} - 6 x ^{2} + 3 .

Note that this expression is not guaranteed to be positive, and is therefore not a valid probability distribution. The Gram–Charlier A series diverges in many cases of interest—it converges only if

f(x) falls off faster than \exp( - (x ^{2)/4)} at infinity (Cramér 1957). When it does not converge, the series is also not a true asymptotic expansion, because it is not possible to estimate the error of the expansion. For this reason, the Edgeworth series (see next section) is generally preferred over the Gram–Charlier A series.

The Edgeworth series

Edgeworth developed a similar expansion as an improvement to the central limit theorem.^[4] The advantage of the Edgeworth series is that the error is controlled, so that it is a true asymptotic expansion.

Let

\{Z _{i\}} be a sequence of independent and identically distributed random variables with finite mean \mu and variance \sigma ^{2} , and let X _{n} be their standardized sums:X _{n} = (1)/(\sqrt{n) } \sum_^n \frac.

Let

F _{n} denote the cumulative distribution functions of the variables X _{n} . Then by the central limit theorem,\lim _{n\toinfty} F _{n(x)} = \Phi(x) \equiv \int ^{x} _{-inf ty} \tfrac{ 1 }{\sqrt{ 2 \pi}}e ^{- (1)/(q ^{2} 2)} dq

for every

x , as long as the mean and variance are finite.

The standardization of

\{Z _{i\}} ensures that the first two cumulants of X _{n} are \kappa ^{F _{n}} _{1} = 0 and \kappa ^{F _{n}} _{2} = 1 . Now assume that, in addition to having mean \mu and variance \sigma ^{2} , the i.i.d. random variables Z _{i} have higher cumulants \kappa _{r} . From the additivity and homogeneity properties of cumulants, the cumulants of X _{n} in terms of the cumulants of Z _{i} are for r \geq 2 , \kappa ^{F _{n}} _{r} = (n \kappa _{r})/(\sigma ^{r} n ^{r/2}) = (λ _{r})/(n ^{r/2}) where λ _{r} = (\kappa _{r})/(\sigma ^{r}) .

If we expand the formal expression of the characteristic function

\hat{f} _{n(t)} of F _{n} in terms of the standard normal distribution, that is, if we set\phi(x) = (1)/(\sqrt{ 2 \pi) }\exp(-\tfracx^2),

then the cumulant differences in the expansion are

\kappa ^{F _{n}} _{1- \gamma} _{1} = 0 , \kappa ^{F _{n}} _{2- \gamma} _{2} = 0 , \kappa ^{F _{n}} _{r - \gamma} _{r} = (λ _{r})/(n ^{r/2-1}) ; r\geq 3 .

The Gram–Charlier A series for the density function of

X _{n} is nowf _{n(x)} = \phi(x) \sum ^{inf ty} _{r =0} (1)/(r!) B _{r} \left( 0 , 0 , (λ _{3})/(,\ldots, n ^{1/2}) (λ _{r})/(n ^{r/2-1}) \right) He _{r(x).}

The Edgeworth series is developed similarly to the Gram–Charlier A series, only that now terms are collected according to powers of

n . The coefficients of n^−m/2 term can be obtained by collecting the monomials of the Bell polynomials corresponding to the integer partitions of m. Thus, we have the characteristic function as\hat{f} _{n(t)=\left[1+\sum} ^{inf ty} ^{j =1} (P _{j(it)})/(n ^{j/2}) \right] \exp( - t ^{2/2),}

where

P _{j(x)} is a polynomial of degree 3 j . Again, after inverse Fourier transform, the density function f _{n} follows asf _{n(x)} = \phi(x) + \sum ^{inf ty} _{j =1} (P _{j(-D)})/(n ^{j/2}) \phi(x).

Likewise, integrating the series, we obtain the distribution function

F _{n(x)} = \Phi(x) + \sum ^{inf ty} _{j =1} (1)/(n ^{j/2}) (P _{j(-D)})/(D) \phi(x).

We can explicitly write the polynomial

P _{m(-D)} as P _{m(-D)} = \sum \prod _{i} (1)/(\left( k _{i!}) (λ _{l _{i}})/(l _{i!}) \right) ^{k _{i}} ( - D) ^{s,}

where the summation is over all the integer partitions of m such that

\sum _{i} i k _{i} = m and l _{i} = i +2 and s = \sum _{i} k _{i} l _{i.}

For example, if m = 3, then there are three ways to partition this number: 1 + 1 + 1 = 2 + 1 = 3. As such we need to examine three cases:

• 1 + 1 + 1 = 1 · k₁, so we have k₁ = 3, l₁ = 3, and s = 9.
• 1 + 2 = 1 · k₁ + 2 · k₂, so we have k₁ = 1, k₂ = 1, l₁ = 3, l₂ = 4, and s = 7.
• 3 = 3 · k₃, so we have k₃ = 1, l₃ = 5, and s = 5.

Thus, the required polynomial is

\begin{align} P _{3(-D)} & = (1)/(\left( 3 !) (λ _{3})/(3 !) \right) ^{3} ( - D) ^{9} + (1)/(\left( 1 ! 1 !) (λ _{3})/(3 !) \right) \left( (λ _{4})/(4 !) \right) ( - D) ^{7} + (1)/(\left( 1 !) (λ _{5})/(5 !) \right) ( - D) ^{5} \\ & = (λ ^{3} _{3})/(1296) ( - D) ^{9} + (λ _{3} λ _{4})/(144) ( - D) ^{7} + (λ _{5})/(120) ( - D) ^{5. \end{align}}

The first five terms of the expansion are

\begin{align} f _{n(x)} & = \phi(x) \\ & - (1)/(n ^{(1)/(2)}) \left(\tfrac{ 1 }{ 6 } λ ^{( 3 )} _{3 \phi} (x) \right) \\ & + (1)/(n) \left(\tfrac{ 1 }{ 24 } λ ^{( 4 )} _{4 \phi} (x) + \tfrac{ 1 }{ 72 } λ ^{2 \phi} _{3} ^{(6)} (x) \right) \\ & - (1)/(n ^{(3)/(2)}) \left(\tfrac{ 1 }{ 120 } λ ^{( 5 )} _{5 \phi} (x) + \tfrac{ 1 }{ 144 } λ _{3λ} ^{( 7 )} ^{4 \phi} (x) + \tfrac{ 1 }{ 1296 } λ ^{3 \phi} _{3} ^{(9)} (x)\right) \\ & + (1)/(n ^{2}) \left(\tfrac{ 1 }{ 720 } λ ^{( 6 )} _{6 \phi} (x) + \left(\tfrac{ 1 }{ 1152 } λ ^{2} _{4} + \tfrac{ 1 }{ 720 } λ _{3λ} ^{( 8 )} ^{5 \right)\phi} (x) + \tfrac{ 1 }{ 1728 } λ ^{( 10 )} _{4 \phi} (x) + \tfrac{ 1 }{ 31104 } λ ^{4 \phi} _{3} ^{(12)} (x) \right)\\ & + O \left (n ^{- (5)/(2)} \right ). \end{align}

Here, is the j-th derivative of at point x. Remembering that the derivatives of the density of the normal distribution are related to the normal density by

\phi ^{(n)} (x) = ( -1 ) ^{n} He _{n(x)\phi(x)} , (where He _{n} is the Hermite polynomial of order n), this explains the alternative representations in terms of the density function. Blinnikov and Moessner (1998) have given a simple algorithm to calculate higher-order terms of the expansion.

Note that in case of a lattice distributions (which have discrete values), the Edgeworth expansion must be adjusted to account for the discontinuous jumps between lattice points.^[5]

Illustration: density of the sample mean of three χ² distributions

Take

X _{i} \sim \chi ^{2(k=2),} i =1 , 2 , 3 (n =3 ) and the sample mean \bar X = (1)/(3) \sum ^{3} _{i =1} X _{i} .

We can use several distributions for

\bar X :

• The exact distribution, which follows a gamma distribution:

\bar X \sim Gamma \left(\alpha = n ⋅ k / 2 , \theta = 2 /n \right) = Gamma \left(\alpha =3 , \theta = 2 / 3 \right) .

• The asymptotic normal distribution:

\bar X \xrightarrow{n \to inf ty} N(k, 2 ⋅ k /n ) = N( 2 , 4 / 3 ) .

• Two Edgeworth expansions, of degrees 2 and 3.

Discussion of results

• For finite samples, an Edgeworth expansion is not guaranteed to be a proper probability distribution as the CDF values at some points may go beyond

[ 0 , 1 ] .

• They guarantee (asymptotically) absolute errors, but relative errors can be easily assessed by comparing the leading Edgeworth term in the remainder with the overall leading term.

See also

• Cornish–Fisher expansion
• Edgeworth binomial tree

Further reading

• H. Cramér. (1957). Mathematical Methods of Statistics. Princeton University Press, Princeton.
• Wallace . D. L. . 1958 . Asymptotic approximations to distributions . Annals of Mathematical Statistics . 29 . 3. 635–654 . 10.1214/aoms/1177706528. free .
• M. Kendall & A. Stuart. (1977), The advanced theory of statistics, Vol 1: Distribution theory, 4th Edition, Macmillan, New York.
• P. McCullagh (1987). Tensor Methods in Statistics. Chapman and Hall, London.
• D. R. Cox and O. E. Barndorff-Nielsen (1989). Asymptotic Techniques for Use in Statistics. Chapman and Hall, London.
• P. Hall (1992). The Bootstrap and Edgeworth Expansion. Springer, New York.
  • Blinnikov . S. . Moessner . R. . 1998 . Expansions for nearly Gaussian distributions . Astronomy and Astrophysics Supplement Series . 130 . 193–205 . 10.1051/aas:1998221. astro-ph/9711239 . 1998A&AS..130..193B .
• Martin. Douglas . Arora. Rohit . 2017 . Inefficiency and bias of modified value-at-risk and expected shortfall . Journal of Risk . 19 . 6 . 59–84 . 10.21314/JOR.2017.365.
• J. E. Kolassa (2006). Series Approximation Methods in Statistics (3rd ed.). (Lecture Notes in Statistics #88). Springer, New York.

Notes and References

1. Stuart, A., & Kendall, M. G. (1968). The advanced theory of statistics. Hafner Publishing Company.
2. Book: Kolassa . John E. . Series approximation methods in statistics . 2006 . Springer . 0387322272 . 3rd.
3. Wallace . D. L. . 1958 . Asymptotic Approximations to Distributions . Annals of Mathematical Statistics . 29 . 3 . 635–654 . 2237255 . 10.1214/aoms/1177706528. free .
4. Hall, P. (2013). The bootstrap and Edgeworth expansion. Springer Science & Business Media.
5. Kolassa . John E. . Peter . McCullagh . Edgeworth series for lattice distributions . . 18 . 2 . 1990 . 981–985 . 2242145 . 10.1214/aos/1176347637. free .