Neyman–Pearson lemma

In statistics, the Neyman–Pearson lemma, named after Jerzy Neyman and Egon Pearson, states that when performing a hypothesis test between two point hypotheses H₀: θ = θ₀ and H₁: θ = θ₁, then the likelihood-ratio test which rejects H₀ in favour of H₁ when

$\Lambda (x)={\frac {L(\theta _{0}\mid x)}{L(\theta _{1}\mid x)}}\leq \eta$

where

$P(\Lambda (X)\leq \eta \mid H_{0})=\alpha$

is the most powerful test of size α for a threshold η. If the test is most powerful for all $\theta _{1}\in \Theta _{1}$, it is said to be uniformly most powerful (UMP) for alternatives in the set $\Theta _{1}\,$.

In practice, the likelihood ratio is often used directly to construct tests — see Likelihood-ratio test. However it can also be used to suggest particular test-statistics that might be of interest or to suggest simplified tests — for this one considers algebraic manipulation of the ratio to see if there are key statistics in it related to the size of the ratio (i.e. whether a large statistic corresponds to a small ratio or to a large one).

Proof

Define the rejection region of the null hypothesis for the NP test as

$R_{NP}=\left\{x:{\frac {L(\theta _{0}|x)}{L(\theta _{1}|x)}}\leq \eta \right\}.$

Any other test will have a different rejection region that we define as $R_{A}$. Furthermore, define the probability of the data falling in region R, given parameter $\theta$ as

$P(R,\theta )=\int _{R}L(\theta |x)\,dx,$

For both tests to have size $\alpha$, it must be true that

$\alpha =P(R_{NP},\theta _{0})=P(R_{A},\theta _{0})\,.$

It will be useful to break these down into integrals over distinct regions:

$P(R_{NP},\theta )=P(R_{NP}\cap R_{A},\theta )+P(R_{NP}\cap R_{A}^{c},\theta ),$

and

$P(R_{A},\theta )=P(R_{NP}\cap R_{A},\theta )+P(R_{NP}^{c}\cap R_{A},\theta ).$

Setting $\theta =\theta _{0}$ and equating the above two expression yields that

$P(R_{NP}\cap R_{A}^{c},\theta _{0})=P(R_{NP}^{c}\cap R_{A},\theta _{0}).$

Comparing the powers of the two tests, $P(R_{NP},\theta _{1})$ and $P(R_{A},\theta _{1})$, one can see that

$P(R_{NP},\theta _{1})\geq P(R_{A},\theta _{1})\iff P(R_{NP}\cap R_{A}^{c},\theta _{1})\geq P(R_{NP}^{c}\cap R_{A},\theta _{1}).$

Now by the definition of $R_{NP}$,

$P(R_{NP}\cap R_{A}^{c},\theta _{1})=\int _{R_{NP}\cap R_{A}^{c}}L(\theta _{1}|x)\,dx\geq {\frac {1}{\eta }}\int _{R_{NP}\cap R_{A}^{c}}L(\theta _{0}|x)\,dx={\frac {1}{\eta }}P(R_{NP}\cap R_{A}^{c},\theta _{0})$

$={\frac {1}{\eta }}P(R_{NP}^{c}\cap R_{A},\theta _{0})={\frac {1}{\eta }}\int _{R_{NP}^{c}\cap R_{A}}L(\theta _{0}|x)\,dx\geq \int _{R_{NP}^{c}\cap R_{A}}L(\theta _{1}|x)dx=P(R_{NP}^{c}\cap R_{A},\theta _{1}).$

Hence the inequality holds.

Example

Let $X_{1},\dots ,X_{n}$ be a random sample from the ${\mathcal {N}}(\mu ,\sigma ^{2})$ distribution where the mean $\mu$ is known, and suppose that we wish to test for $H_{0}:\sigma ^{2}=\sigma _{0}^{2}$ against $H_{1}:\sigma ^{2}=\sigma _{1}^{2}$. The likelihood for this set of normally distributed data is

$L\left(\sigma ^{2};\mathbf {x} \right)\propto \left(\sigma ^{2}\right)^{-n/2}\exp \left\{-{\frac {\sum _{i=1}^{n}\left(x_{i}-\mu \right)^{2}}{2\sigma ^{2}}}\right\}.$

We can compute the likelihood ratio to find the key statistic in this test and its effect on the test's outcome:

$\Lambda (\mathbf {x} )={\frac {L\left(\sigma _{0}^{2};\mathbf {x} \right)}{L\left(\sigma _{1}^{2};\mathbf {x} \right)}}=\left({\frac {\sigma _{0}^{2}}{\sigma _{1}^{2}}}\right)^{-n/2}\exp \left\{-{\frac {1}{2}}(\sigma _{0}^{-2}-\sigma _{1}^{-2})\sum _{i=1}^{n}\left(x_{i}-\mu \right)^{2}\right\}.$

This ratio only depends on the data through $\sum _{i=1}^{n}\left(x_{i}-\mu \right)^{2}$. Therefore, by the Neyman–Pearson lemma, the most powerful test of this type of hypothesis for this data will depend only on $\sum _{i=1}^{n}\left(x_{i}-\mu \right)^{2}$. Also, by inspection, we can see that if $\sigma _{1}^{2}>\sigma _{0}^{2}$, then $\Lambda (\mathbf {x} )$ is a decreasing function of $\sum _{i=1}^{n}\left(x_{i}-\mu \right)^{2}$. So we should reject $H_{0}$ if $\sum _{i=1}^{n}\left(x_{i}-\mu \right)^{2}$ is sufficiently large. The rejection threshold depends on the size of the test. In this example, the test statistic can be shown to be a scaled Chi-square distributed random variable and an exact critical value can be obtained.

See also

• Statistical power

References

• cnx.org: Neyman–Pearson criterion

External links

• Cosma Shalizi, a professor of statistics at Carnegie Mellon University, gives an intuitive derivation of the Neyman–Pearson Lemma using ideas from economics