Hodges' estimator explained

In statistics, Hodges' estimator (or the Hodges–Le Cam estimator), named for Joseph Hodges, is a famous counterexample of an estimator which is "superefficient", i.e. it attains smaller asymptotic variance than regular efficient estimators. The existence of such a counterexample is the reason for the introduction of the notion of regular estimators.

Hodges' estimator improves upon a regular estimator at a single point. In general, any superefficient estimator may surpass a regular estimator at most on a set of Lebesgue measure zero.

Although Hodges discovered the estimator he never published it; the first publication was in the doctoral thesis of Lucien Le Cam.^[1]

Construction

Suppose

\hat{\theta} _{n} is a "common" estimator for some parameter \theta : it is consistent, and converges to some asymptotic distribution L _{\theta} (usually this is a normal distribution with mean zero and variance which may depend on \theta ) at the \sqrt{n} -rate:\sqrt{n}(\hat\theta _{n} - \theta) \xrightarrow{d} L _{\theta .}

Then the Hodges' estimator

\hat{\theta} ^{H} _{n} is defined as\hat\theta ^{H} _{n} = \begin{cases}\hat\theta _{n,} & if |\hat\theta _{n|} \geq n ^{-1/4} , and \ 0 , & if |\hat\theta _{n|} < n ^{-1/4} .\end{cases} This estimator is equal to \hat{\theta} _{n} everywhere except on the small interval [ - n ^{-1/4} ,n ^{-1/4} ] , where it is equal to zero. It is not difficult to see that this estimator is consistent for \theta , and its asymptotic distribution is\begin{align} & n ^{H} _{n} - \theta) \xrightarrow{d} 0 , when \theta = 0 , \\ &\sqrt{n}(\hat\theta ^{H} _{n} - \theta) \xrightarrow{d} L _{\theta,} when \theta ≠ 0 , \end{align} for any \alpha\in R . Thus this estimator has the same asymptotic distribution as \hat{\theta} _{n} for all \theta ≠ 0 , whereas for \theta =0 the rate of convergence becomes arbitrarily fast. This estimator is superefficient, as it surpasses the asymptotic behavior of the efficient estimator \hat{\theta} _{n} at least at one point \theta =0 .

It is not true that the Hodges estimator is equivalent to the sample mean, but much better when the true mean is 0. The correct interpretation is that, for finite

n , the truncation can lead to worse square error than the sample mean estimator for E[X] close to 0, as is shown in the example in the following section.^[2]

Le Cam shows that this behaviour is typical: superefficiency at the point θ implies the existence of a sequence

\theta _{n} → \theta such that \lim inf E \theta _{n} \ell (\sqrt n (\hat \theta _{n} - \theta _{n} )) is strictly larger than the Cramér-Rao bound. For theextreme case where the asymptotic risk at θ is zero, the \liminf is even infinite for a sequence \theta _{n} → \theta .^[3]

In general, superefficiency may only be attained on a subset of Lebesgue measure zero of the parameter space

\Theta .^[4]

Example

Suppose x₁, ..., x_n is an independent and identically distributed (IID) random sample from normal distribution with unknown mean but known variance. Then the common estimator for the population mean θ is the arithmetic mean of all observations: $\backslash scriptstyle\backslash bar$. The corresponding Hodges' estimator will be $\backslash scriptstyle\backslash hat\backslash theta^H\_n\; \backslash ;=\backslash ;\; \backslash bar\backslash cdot\backslash mathbf\backslash$

\,\geq\,n^\

, where 1 denotes the indicator function.

The mean square error (scaled by n) associated with the regular estimator

x is constant and equal to 1 for all θs. At the same time the mean square error of the Hodges' estimator $\backslash scriptstyle\backslash hat\backslash theta\_n^H$ behaves erratically in the vicinity of zero, and even becomes unbounded as . This demonstrates that the Hodges' estimator is not regular, and its asymptotic properties are not adequately described by limits of the form (θ fixed,).

See also

• James–Stein estimator

Notes

References

• Book: Bickel . Peter J. . Klaassen . Chris A.J. . Ritov . Ya’acov . Wellner . Jon A. . 1998 . Efficient and adaptive estimation for semiparametric models . Springer: New York . 0-387-98473-9 . CITEREFBickel1998 .
• Kale . B.K. . 1985 . A note on the super efficient estimator . Journal of Statistical Planning and Inference . 12 . 259–263 . 10.1016/0378-3758(85)90074-6 .
• Stoica . P. . Ottersten . B. . 1996 . The evil of superefficiency . Signal Processing . 55 . 133–136 . 10.1016/S0165-1684(96)00159-4 .
• Book: Vaart, A. W. van der . 1998 . Asymptotic statistics . Cambridge University Press . 978-0-521-78450-4 .

Notes and References

1. Book: Le Cam, Lucien M. . On some asymptotic properties of maximum likelihood estimates and related Bayes' estimates. . University of California . Berkeley. . 1953 . University of California press . University of California publications in statistics; v. 1, no. 11 . Berkeley.
2. Vaart AW van der. Asymptotic Statistics. Cambridge University Press; 1998.
3. van der Vaart, A. W., & Wellner, J. A. (1996). Weak Convergence and Empirical Processes. In Springer Series in Statistics. Springer New York. https://doi.org/10.1007/978-1-4757-2545-2
4. Vaart AW van der. Asymptotic Statistics. Cambridge University Press; 1998.