# Jackknife resampling

(Redirected from Jackknife method)

In statistics, the jackknife is a resampling technique especially useful for variance and bias estimation. The jackknife predates other common resampling methods such as the bootstrap. The jackknife estimator of a parameter is found by systematically leaving out each observation from a dataset and calculating the estimate and then finding the average of these calculations. Given a sample of size ${\displaystyle N}$, the jackknife estimate is found by aggregating the estimates of each ${\displaystyle N-1}$-sized sub-sample.

The jackknife technique was developed by Maurice Quenouille (1949, 1956). John Tukey (1958) expanded on the technique and proposed the name "jackknife" since, like a physical jack-knife (a compact folding knife), it is a rough-and-ready tool that can improvise a solution for a variety of problems even though specific problems may be more efficiently solved with a purpose-designed tool.[1]

The jackknife is a linear approximation of the bootstrap.[1]

## Estimation

The jackknife estimate of a parameter can be found by estimating the parameter for each subsample omitting the ith observation to estimate the previously unknown value of a parameter (say ${\displaystyle {\bar {x}}_{i}}$).[2]

${\displaystyle {\bar {x}}_{i}={\frac {1}{n-1}}\sum _{j\neq i}^{n}x_{j}}$

## Variance estimation

An estimate of the variance of an estimator can be calculated using the jackknife technique.

${\displaystyle \operatorname {Var} _{\mathrm {(jackknife)} }={\frac {n-1}{n}}\sum _{i=1}^{n}({\bar {x}}_{i}-{\bar {x}}_{\mathrm {(.)} })^{2}}$

where ${\displaystyle {\bar {x}}_{i}}$ is the parameter estimate based on leaving out the ith observation, and ${\displaystyle {\bar {x}}_{\mathrm {(.)} }={\frac {1}{n}}\sum _{i}^{n}{\bar {x}}_{i}}$ is the estimator based on all of the subsamples.[3][4]

## Bias estimation and correction

The jackknife technique can be used to estimate the bias of an estimator calculated over the entire sample. Say ${\displaystyle {\hat {\theta }}}$ is the calculated estimator of the parameter of interest based on all ${\displaystyle {n}}$ observations. Let

${\displaystyle {\hat {\theta }}_{\mathrm {(.)} }={\frac {1}{n}}\sum _{i=1}^{n}{\hat {\theta }}_{\mathrm {(i)} }}$

where ${\displaystyle {\hat {\theta }}_{\mathrm {(i)} }}$ is the estimate of interest based on the sample with the ith observation removed, and ${\displaystyle {\hat {\theta }}_{\mathrm {(.)} }}$ is the average of these "leave-one-out" estimates. The jackknife estimate of the bias of ${\displaystyle {\hat {\theta }}}$ is given by:

${\displaystyle {\widehat {\text{Bias}}}_{\mathrm {(\theta )} }=(n-1)({\hat {\theta }}_{\mathrm {(.)} }-{\hat {\theta }})}$

and the resulting bias-corrected jackknife estimate of ${\displaystyle \theta }$ is given by:

${\displaystyle {\hat {\theta }}_{\text{Jack}}=n{\hat {\theta }}-(n-1){\hat {\theta }}_{\mathrm {(.)} }}$

This removes the bias in the special case that the bias is ${\displaystyle O(N^{-1})}$ and to ${\displaystyle O(N^{-2})}$ in other cases.[1]

This provides an estimated correction of bias due to the estimation method. The jackknife does not correct for a biased sample.

## Notes

1. ^ a b c Cameron & Trivedi 2005, p. 375.
2. ^ Efron 1982, p. 2.
3. ^ Efron 1982, p. 14.
4. ^ McIntosh, Avery I. "The Jackknife Estimation Method" (PDF). Boston University. Avery I. McIntosh. Retrieved 2016-04-30.