Hodges' estimator

In statistics, Hodges' estimator (or the Hodges–Le Cam estimator), named for Joseph Hodges, is a famous counterexample of an estimator which is "superefficient", i.e. it attains smaller asymptotic variance than regular efficient estimators. The existence of such a counterexample is the reason for the introduction of the notion of regular estimators.

Hodges' estimator improves upon a regular estimator at a single point. In general, any superefficient estimator may surpass a regular estimator at most on a set of Lebesgue measure zero.

Construction

Suppose $\scriptstyle {\hat {\theta }}_{n}$ is a "common" estimator for some parameter θ: it is consistent, and converges to some asymptotic distribution L_θ (usually this is a normal distribution with mean zero and variance which may depend on θ) at the √n-rate:

${\sqrt {n}}({\hat {\theta }}_{n}-\theta )\ {\xrightarrow {d}}\ L_{\theta }\ .$

Then the Hodges' estimator $\scriptstyle {\hat {\theta }}_{n}^{H}$ is defined as

${\hat {\theta }}_{n}^{H}={\begin{cases}{\hat {\theta }}_{n},&{\text{if }}|{\hat {\theta }}_{n}|\geq n^{-1/4},{\text{ and}}\\0,&{\text{if }}|{\hat {\theta }}_{n}|<n^{-1/4}.\end{cases}}$

This estimator is equal to $\scriptstyle {\hat {\theta }}_{n}$ everywhere except on the small interval [−n^−1/4, n^−1/4], where it is equal to zero. It is not difficult to see that this estimator is consistent for θ, and its asymptotic distribution is

${\begin{aligned}&n^{\alpha }({\hat {\theta }}_{n}^{H}-\theta )\ {\xrightarrow {d}}\ 0,\qquad {\text{when }}\theta =0,\\&{\sqrt {n}}({\hat {\theta }}_{n}^{H}-\theta )\ {\xrightarrow {d}}\ L_{\theta },\quad {\text{when }}\theta \neq 0,\end{aligned}}$

for any α ∈ R. Thus this estimator has the same asymptotic distribution as $\scriptstyle {\hat {\theta }}_{n}$ for all θ ≠ 0, whereas for θ = 0 the rate of convergence becomes arbitrarily fast. This estimator is superefficient, as it surpasses the asymptotic behavior of the efficient estimator $\scriptstyle {\hat {\theta }}_{n}$ at least at one point θ = 0. In general, superefficiency may only be attained on a subset of Lebesgue measure zero of the parameter space Θ.

Example

The mean square error (times n) of Hodges' estimator. Blue curve corresponds to n = 5, purple to n = 50, and olive to n = 500.

Suppose x₁, ..., x_n is an independent and identically distributed (IID) random sample from normal distribution N(θ, 1) with unknown mean but known variance. Then the common estimator for the population mean θ is the arithmetic mean of all observations: $\scriptstyle {\bar {x}}$. The corresponding Hodges' estimator will be $\scriptstyle {\hat {\theta }}_{n}^{H}\;=\;{\bar {x}}\cdot \mathbf {1} \{|{\bar {x}}|\,\geq \,n^{-1/4}\}$, where 1{...} denotes the indicator function.

The mean square error (scaled by n) associated with the regular estimator x is constant and equal to 1 for all θ's. At the same time the mean square error of the Hodges' estimator $\scriptstyle {\hat {\theta }}_{n}^{H}$ behaves erratically in the vicinity of zero, and even becomes unbounded as n → ∞. This demonstrates that the Hodges' estimator is not regular, and its asymptotic properties are not adequately described by limits of the form (θ fixed, n → ∞).

See also

• James–Stein estimator

Notes

References

• Bickel, Peter J.; Klaassen, Chris A.J.; Ritov, Ya’acov; Wellner, Jon A. (1998). Efficient and adaptive estimation for semiparametric models. Springer: New York. ISBN 0-387-98473-9.
• Kale, B.K. (1985). "A note on the super efficient estimator". Journal of Statistical Planning and Inference. 12: 259–263. doi:10.1016/0378-3758(85)90074-6. {{cite journal}}: Invalid |ref=harv (help)
• Stoica, P.; Ottersten, B. (1996). "The evil of superefficiency". Signal Processing. 55: 133–136. doi:10.1016/S0165-1684(96)00159-4. {{cite journal}}: Invalid |ref=harv (help)
• Vaart, A. W. van der (1998). Asymptotic statistics. Cambridge University Press. ISBN 978-0-521-78450-4. {{cite book}}: Invalid |ref=harv (help)