Poisson limit theorem

"Poisson theorem" redirects here. For the "Poisson's theorem" in Hamiltonian mechanics, see Poisson bracket § Constants of motion.

The law of rare events or Poisson limit theorem gives a Poisson approximation to the binomial distribution, under certain conditions. The theorem was named after Siméon Denis Poisson (1781–1840).

Statement

If

$n\rightarrow \infty ,p\rightarrow 0$, such that $np\rightarrow \lambda$

then

${\frac {n!}{(n-k)!k!}}p^{k}(1-p)^{n-k}\rightarrow e^{-\lambda }{\frac {\lambda ^{k}}{k!}}.$

Example

Suppose that in an interval [0, 1000], 500 points are placed randomly. Now what is the number of points that will be placed in [0, 10]?

The probabilistically precise way of describing the number of points in the sub-interval would be to describe it as a binomial distribution $p_{n}(k)$.

If we look here, the probability (that a random point will be placed in the sub-interval) is $p=10/1000=0.01$. Here $n=500$ so $np=5$.

The probability that $k$ points lie in the sub-interval is

$p_{n}(k)={\frac {n!}{(n-k)!k!}}p^{k}(1-p)^{n-k}.$

where: $p$ is the probability of falling with in the interval. $n!/(k!\cdot (n-k)!)$ gives the number of ways in which $k$ elements can be selected. $p^{k}$ gives the probability of the $k$ elements falling in the interval. $(1-p)^{n-k}$ counts the probability that ${n-k}$ elements fall outside of the interval

But using the Poisson Theorem we can approximate it as

$e^{-\lambda }{\frac {\lambda ^{k}}{k!}}=e^{-5}{\frac {5^{k}}{k!}}.$

Proofs

According to factorial's rate of growth, we replace factorials of large numbers with approximations:

${\frac {n!}{(n-k)!k!}}p^{k}(1-p)^{n-k}\rightarrow {\frac {{\sqrt {2\pi n}}\left({\frac {n}{e}}\right)^{n}}{{\sqrt {2\pi \left(n-k\right)}}\left({\frac {n-k}{e}}\right)^{n-k}k!}}p^{k}(1-p)^{n-k}.$

After simplifying the fraction:

${\frac {{\sqrt {2\pi n}}\left({\frac {n}{e}}\right)^{n}}{{\sqrt {2\pi \left(n-k\right)}}\left({\frac {n-k}{e}}\right)^{n-k}k!}}p^{k}(1-p)^{n-k}\rightarrow {\frac {{\sqrt {n}}n^{n}p^{k}(1-p)^{n-k}}{{\sqrt {n-k}}\left(n-k\right)^{n-k}e^{k}k!}}\rightarrow {\frac {n^{n}p^{k}(1-p)^{n-k}}{\left(n-k\right)^{n-k}e^{k}k!}}.$

After using the condition $np\rightarrow \lambda$:

${\frac {n^{n}p^{k}(1-p)^{n-k}}{\left(n-k\right)^{n-k}e^{k}k!}}\rightarrow {\frac {n^{k}\left({\frac {\lambda }{n}}\right)^{k}(1-{\frac {\lambda }{n}})^{n-k}}{\left(1-{\frac {k}{n}}\right)^{n-k}e^{k}k!}}={\frac {\lambda ^{k}\left(1-{\frac {\lambda }{n}}\right)^{n-k}}{\left(1-{\frac {k}{n}}\right)^{n-k}e^{k}k!}}\rightarrow {\frac {\lambda ^{k}\left(1-{\frac {\lambda }{n}}\right)^{n}}{\left(1-{\frac {k}{n}}\right)^{n}e^{k}k!}}$

As $n\rightarrow \infty$, $\left(1+{\frac {x}{n}}\right)^{n}\rightarrow e^{x}$ so:

${\frac {\lambda ^{k}\left(1-{\frac {\lambda }{n}}\right)^{n}}{\left(1-{\frac {k}{n}}\right)^{n}e^{k}k!}}\rightarrow {\frac {\lambda ^{k}e^{-\lambda }}{e^{-k}e^{k}k!}}={\frac {\lambda ^{k}e^{-\lambda }}{k!}}$

Q.E.D.

Alternative Proof

If we make the stronger assumption $np=\lambda$ (rather than $np\rightarrow \lambda$) then a simpler proof is possible without needing approximations for the factorials. Since $np=\lambda$, we can rewrite $p=\lambda /n$. We now have:

$\lim _{n\to \infty }{\frac {n!}{(n-k)!k!}}\left({\frac {\lambda }{n}}\right)^{k}\left(1-{\frac {\lambda }{n}}\right)^{n-k}=\lim _{n\to \infty }{\frac {n(n-1)(n-2)\dots (n-k+1)}{k!}}{\frac {\lambda ^{k}}{n^{k}}}\left(1-{\frac {\lambda }{n}}\right)^{n-k}$

Taking each of these terms in sequence, $n(n-1)(n-2)\dots (n-k+1)=n^{k}+O\left(n^{k-1}\right)$, meaning that $\lim _{n\to \infty }{\frac {n(n-1)(n-2)\dots (n-k+1)}{n^{k}k!}}={\frac {1}{k!}}$.

Now $\left(1-{\frac {\lambda }{n}}\right)^{n-k}=\left(1-{\frac {\lambda }{n}}\right)^{n}\left(1-{\frac {\lambda }{n}}\right)^{-k}$. The first portion of this converges to $e^{-\lambda }$, and the second portion goes to 1, as $\lim _{n\to \infty }\left(1-{\frac {\lambda }{n}}\right)^{-k}=\lim _{n\to \infty }\left(1-0\right)^{-k}=1$

This leaves us with ${\frac {1}{k!}}\lambda ^{k}e^{-\lambda }$. Q.E.D.

Ordinary Generating Functions

It is also possible to demonstrate the theorem through the use of Ordinary Generating Functions (OGF). Indeed, the OGF of the binomial distribution is

$G_{\mathrm {bin} }(x;p,N)\equiv \sum _{k=0}^{N}\left[{\binom {N}{k}}p^{k}(1-p)^{N-k}\right]x^{k}={\Big [}1+(x-1)p{\Big ]}^{N}$

by virtue of the Binomial Theorem. Taking the limit $N\rightarrow \infty$ while keeping the product $pN\equiv \lambda$ constant, we find

$\lim _{N\rightarrow \infty }G_{\mathrm {bin} }(x;p,N)=\lim _{N\rightarrow \infty }{\Big [}1+{\frac {\lambda (x-1)}{N}}{\Big ]}^{N}=\mathrm {e} ^{\lambda (x-1)}=\sum _{k=0}^{\infty }\left[{\frac {\mathrm {e} ^{-\lambda }\lambda ^{k}}{k!}}\right]x^{k}$

which is the OGF for the Poisson distribution. (The second equality holds due to the definition of the Exponential function.)

See also

• De Moivre–Laplace theorem
• Le Cam's theorem