Poisson limit theorem

"Poisson theorem" redirects here. For the "Poisson's theorem" in Hamiltonian mechanics, see Poisson bracket § Constants of motion.

The law of rare events or Poisson limit theorem gives a Poisson approximation to the binomial distribution, under certain conditions.[1] The theorem was named after Siméon Denis Poisson (1781–1840).

Statement

If

${\displaystyle n\rightarrow \infty ,p\rightarrow 0}$, such that ${\displaystyle np\rightarrow \lambda }$

then

${\displaystyle {\frac {n!}{(n-k)!k!}}p^{k}(1-p)^{n-k}\rightarrow e^{-\lambda }{\frac {\lambda ^{k}}{k!}}.}$

Example

Suppose that in an interval [0, 1000], 500 points are placed randomly. Now what is the number of points that will be placed in [0, 10]?

The probabilistically precise way of describing the number of points in the sub-interval would be to describe it as a binomial distribution ${\displaystyle p_{n}(k)}$.

If we look here, the probability (that a random point will be placed in the sub-interval) is ${\displaystyle p=10/1000=0.01}$. Here ${\displaystyle n=500}$ so ${\displaystyle np=5}$.

The probability that ${\displaystyle k}$ points lie in the sub-interval is

${\displaystyle p_{n}(k)={\frac {n!}{(n-k)!k!}}p^{k}(1-p)^{n-k}.}$

where: ${\displaystyle p}$ is the probability of falling with in the interval. ${\displaystyle n!/(k!\cdot (n-k)!)}$ gives the number of ways in which ${\displaystyle k}$ elements can be selected. ${\displaystyle p^{k}}$ gives the probability of the ${\displaystyle k}$ elements falling in the interval. ${\displaystyle (1-p)^{n-k}}$ counts the probability that ${\displaystyle {n-k}}$ elements fall outside of the interval

But using the Poisson Theorem we can approximate it as

${\displaystyle e^{-\lambda }{\frac {\lambda ^{k}}{k!}}=e^{-5}{\frac {5^{k}}{k!}}.}$

Proofs

According to factorial's rate of growth, we replace factorials of large numbers with approximations:

${\displaystyle {\frac {n!}{(n-k)!k!}}p^{k}(1-p)^{n-k}\rightarrow {\frac {{\sqrt {2\pi n}}\left({\frac {n}{e}}\right)^{n}}{{\sqrt {2\pi \left(n-k\right)}}\left({\frac {n-k}{e}}\right)^{n-k}k!}}p^{k}(1-p)^{n-k}.}$

After simplifying the fraction:

${\displaystyle {\frac {{\sqrt {2\pi n}}\left({\frac {n}{e}}\right)^{n}}{{\sqrt {2\pi \left(n-k\right)}}\left({\frac {n-k}{e}}\right)^{n-k}k!}}p^{k}(1-p)^{n-k}\rightarrow {\frac {{\sqrt {n}}n^{n}p^{k}(1-p)^{n-k}}{{\sqrt {n-k}}\left(n-k\right)^{n-k}e^{k}k!}}\rightarrow {\frac {n^{n}p^{k}(1-p)^{n-k}}{\left(n-k\right)^{n-k}e^{k}k!}}.}$

After using the condition ${\displaystyle np\rightarrow \lambda }$:

${\displaystyle {\frac {n^{n}p^{k}(1-p)^{n-k}}{\left(n-k\right)^{n-k}e^{k}k!}}\rightarrow {\frac {n^{k}\left({\frac {\lambda }{n}}\right)^{k}(1-{\frac {\lambda }{n}})^{n-k}}{\left(1-{\frac {k}{n}}\right)^{n-k}e^{k}k!}}={\frac {\lambda ^{k}\left(1-{\frac {\lambda }{n}}\right)^{n-k}}{\left(1-{\frac {k}{n}}\right)^{n-k}e^{k}k!}}\rightarrow {\frac {\lambda ^{k}\left(1-{\frac {\lambda }{n}}\right)^{n}}{\left(1-{\frac {k}{n}}\right)^{n}e^{k}k!}}}$

As ${\displaystyle n\rightarrow \infty }$, ${\displaystyle \left(1+{\frac {x}{n}}\right)^{n}\rightarrow e^{x}}$ so:

${\displaystyle {\frac {\lambda ^{k}\left(1-{\frac {\lambda }{n}}\right)^{n}}{\left(1-{\frac {k}{n}}\right)^{n}e^{k}k!}}\rightarrow {\frac {\lambda ^{k}e^{-\lambda }}{e^{-k}e^{k}k!}}={\frac {\lambda ^{k}e^{-\lambda }}{k!}}}$

Q.E.D.

Alternative Proof

If we make the stronger assumption ${\displaystyle np=\lambda }$ (rather than ${\displaystyle np\rightarrow \lambda }$) then a simpler proof is possible without needing approximations for the factorials. Since ${\displaystyle np=\lambda }$, we can rewrite ${\displaystyle p=\lambda /n}$. We now have:

${\displaystyle \lim _{n\to \infty }{\frac {n!}{(n-k)!k!}}\left({\frac {\lambda }{n}}\right)^{k}\left(1-{\frac {\lambda }{n}}\right)^{n-k}=\lim _{n\to \infty }{\frac {n(n-1)(n-2)\dots (n-k+1)}{k!}}{\frac {\lambda ^{k}}{n^{k}}}\left(1-{\frac {\lambda }{n}}\right)^{n-k}}$

Taking each of these terms in sequence, ${\displaystyle n(n-1)(n-2)\dots (n-k+1)=n^{k}+O\left(n^{k-1}\right)}$, meaning that ${\displaystyle \lim _{n\to \infty }{\frac {n(n-1)(n-2)\dots (n-k+1)}{n^{k}k!}}={\frac {1}{k!}}}$.

Now ${\displaystyle \left(1-{\frac {\lambda }{n}}\right)^{n-k}=\left(1-{\frac {\lambda }{n}}\right)^{n}\left(1-{\frac {\lambda }{n}}\right)^{-k}}$. The first portion of this converges to ${\displaystyle e^{-\lambda }}$, and the second portion goes to 1, as ${\displaystyle \lim _{n\to \infty }\left(1-{\frac {\lambda }{n}}\right)^{-k}=\lim _{n\to \infty }\left(1-0\right)^{-k}=1}$

This leaves us with ${\displaystyle {\frac {1}{k!}}\lambda ^{k}e^{-\lambda }}$. Q.E.D.

Ordinary Generating Functions

It is also possible to demonstrate the theorem through the use of Ordinary Generating Functions (OGF). Indeed, the OGF of the binomial distribution is

${\displaystyle G_{\mathrm {bin} }(x;p,N)\equiv \sum _{k=0}^{N}\left[{\binom {N}{k}}p^{k}(1-p)^{N-k}\right]x^{k}={\Big [}1+(x-1)p{\Big ]}^{N}}$

by virtue of the Binomial Theorem. Taking the limit ${\displaystyle N\rightarrow \infty }$ while keeping the product ${\displaystyle pN\equiv \lambda }$ constant, we find

${\displaystyle \lim _{N\rightarrow \infty }G_{\mathrm {bin} }(x;p,N)=\lim _{N\rightarrow \infty }{\Big [}1+{\frac {\lambda (x-1)}{N}}{\Big ]}^{N}=\mathrm {e} ^{\lambda (x-1)}=\sum _{k=0}^{\infty }\left[{\frac {\mathrm {e} ^{-\lambda }\lambda ^{k}}{k!}}\right]x^{k}}$

which is the OGF for the Poisson distribution. (The second equality holds due to the definition of the Exponential function.)