# Coupon collector's problem

In probability theory, the coupon collector's problem refers to mathematical analysis of "collect all coupons and win" contests. It asks the following question: If each box of a brand of cereals contains a coupon, and there are n different types of coupons, what is the probability that more than t boxes need to be bought to collect all n coupons? An alternative statement is: Given n coupons, how many coupons do you expect you need to draw with replacement before having drawn each coupon at least once? The mathematical analysis of the problem reveals that the expected number of trials needed grows as ${\displaystyle \Theta (n\log(n))}$.[a] For example, when n = 50 it takes about 225[b] trials on average to collect all 50 coupons.

## Solution

### Via generating functions

By definition of Stirling numbers of the second kind, the probability that exactly T draws are needed is

${\displaystyle {\frac {S(T-1,n-1)n!}{n^{T}}}}$
By manipulating the generating function of the Stirling numbers, we can explicitly calculate all moments of T:
${\displaystyle f_{k}(x):=\sum _{T}S(T,k)x^{T}=\prod _{r=1}^{k}{\frac {x}{1-rx}}}$
In general, the k-th moment is ${\displaystyle (n-1)!((D_{x}x)^{k}f_{n-1}(x)){\Big |}_{x=1/n}}$, where ${\displaystyle D_{x}}$ is the derivative operator ${\displaystyle d/dx}$. For example, the 0th moment is
${\displaystyle \sum _{T}{\frac {S(T-1,n-1)n!}{n^{T}}}=(n-1)!f_{n-1}(1/n)=(n-1)!\times \prod _{r=1}^{n-1}{\frac {1/n}{1-r/n}}=1}$
and the 1st moment is ${\displaystyle (n-1)!(D_{x}xf_{n-1}(x)){\Big |}_{x=1/n}}$, which can be explicitly evaluated to ${\displaystyle nH_{n}}$, etc.

### Calculating the expectation

Let time T be the number of draws needed to collect all n coupons, and let ti be the time to collect the i-th coupon after i − 1 coupons have been collected. Then ${\displaystyle T=t_{1}+\cdots +t_{n}}$. Think of T and ti as random variables. Observe that the probability of collecting a new coupon is ${\displaystyle p_{i}={\frac {n-(i-1)}{n}}={\frac {n-i+1}{n}}}$. Therefore, ${\displaystyle t_{i}}$ has geometric distribution with expectation ${\displaystyle {\frac {1}{p_{i}}}={\frac {n}{n-i+1}}}$. By the linearity of expectations we have:

{\displaystyle {\begin{aligned}\operatorname {E} (T)&{}=\operatorname {E} (t_{1}+t_{2}+\cdots +t_{n})\\&{}=\operatorname {E} (t_{1})+\operatorname {E} (t_{2})+\cdots +\operatorname {E} (t_{n})\\&{}={\frac {1}{p_{1}}}+{\frac {1}{p_{2}}}+\cdots +{\frac {1}{p_{n}}}\\&{}={\frac {n}{n}}+{\frac {n}{n-1}}+\cdots +{\frac {n}{1}}\\&{}=n\cdot \left({\frac {1}{1}}+{\frac {1}{2}}+\cdots +{\frac {1}{n}}\right)\\&{}=n\cdot H_{n}.\end{aligned}}}

Here Hn is the n-th harmonic number. Using the asymptotics of the harmonic numbers, we obtain:

${\displaystyle \operatorname {E} (T)=n\cdot H_{n}=n\log n+\gamma n+{\frac {1}{2}}+O(1/n),}$

where ${\displaystyle \gamma \approx 0.5772156649}$ is the Euler–Mascheroni constant.

Using the Markov inequality to bound the desired probability:

${\displaystyle \operatorname {P} (T\geq cnH_{n})\leq {\frac {1}{c}}.}$

The above can be modified slightly to handle the case when we've already collected some of the coupons. Let k be the number of coupons already collected, then:

{\displaystyle {\begin{aligned}\operatorname {E} (T_{k})&{}=\operatorname {E} (t_{k+1}+t_{k+2}+\cdots +t_{n})\\&{}=n\cdot \left({\frac {1}{1}}+{\frac {1}{2}}+\cdots +{\frac {1}{n-k}}\right)\\&{}=n\cdot H_{n-k}\end{aligned}}}

And when ${\displaystyle k=0}$ then we get the original result.

### Calculating the variance

Using the independence of random variables ti, we obtain:

{\displaystyle {\begin{aligned}\operatorname {Var} (T)&{}=\operatorname {Var} (t_{1}+\cdots +t_{n})\\&{}=\operatorname {Var} (t_{1})+\operatorname {Var} (t_{2})+\cdots +\operatorname {Var} (t_{n})\\&{}={\frac {1-p_{1}}{p_{1}^{2}}}+{\frac {1-p_{2}}{p_{2}^{2}}}+\cdots +{\frac {1-p_{n}}{p_{n}^{2}}}\\&{}<\left({\frac {n^{2}}{n^{2}}}+{\frac {n^{2}}{(n-1)^{2}}}+\cdots +{\frac {n^{2}}{1^{2}}}\right)\\&{}=n^{2}\cdot \left({\frac {1}{1^{2}}}+{\frac {1}{2^{2}}}+\cdots +{\frac {1}{n^{2}}}\right)\\&{}<{\frac {\pi ^{2}}{6}}n^{2}\end{aligned}}}

since ${\displaystyle {\frac {\pi ^{2}}{6}}={\frac {1}{1^{2}}}+{\frac {1}{2^{2}}}+\cdots +{\frac {1}{n^{2}}}+\cdots }$ (see Basel problem).

Bound the desired probability using the Chebyshev inequality:

${\displaystyle \operatorname {P} \left(|T-nH_{n}|\geq cn\right)\leq {\frac {\pi ^{2}}{6c^{2}}}.}$

### Tail estimates

A stronger tail estimate for the upper tail be obtained as follows. Let ${\displaystyle {Z}_{i}^{r}}$ denote the event that the ${\displaystyle i}$-th coupon was not picked in the first ${\displaystyle r}$ trials. Then

{\displaystyle {\begin{aligned}P\left[{Z}_{i}^{r}\right]=\left(1-{\frac {1}{n}}\right)^{r}\leq e^{-r/n}.\end{aligned}}}

Thus, for ${\displaystyle r=\beta n\log n}$, we have ${\displaystyle P\left[{Z}_{i}^{r}\right]\leq e^{(-\beta n\log n)/n}=n^{-\beta }}$. Via a union bound over the ${\displaystyle n}$ coupons, we obtain

{\displaystyle {\begin{aligned}P\left[T>\beta n\log n\right]=P\left[\bigcup _{i}{Z}_{i}^{\beta n\log n}\right]\leq n\cdot P[{Z}_{1}^{\beta n\log n}]\leq n^{-\beta +1}.\end{aligned}}}

## Extensions and generalizations

${\displaystyle \operatorname {P} (T which is a Gumbel distribution. A simple proof by martingales is in the next section.
• Donald J. Newman and Lawrence Shepp gave a generalization of the coupon collector's problem when m copies of each coupon need to be collected. Let Tm be the first time m copies of each coupon are collected. They showed that the expectation in this case satisfies:
${\displaystyle \operatorname {E} (T_{m})=n\log n+(m-1)n\log \log n+O(n),{\text{ as }}n\to \infty .}$
Here m is fixed. When m = 1 we get the earlier formula for the expectation.
• Common generalization, also due to Erdős and Rényi:
${\displaystyle \operatorname {P} \left(T_{m}
• In the general case of a nonuniform probability distribution, according to Philippe Flajolet et al.[2]
${\displaystyle \operatorname {E} (T)=\int _{0}^{\infty }\left(1-\prod _{i=1}^{m}\left(1-e^{-p_{i}t}\right)\right)dt.}$
This is equal to
${\displaystyle \operatorname {E} (T)=\sum _{q=0}^{m-1}(-1)^{m-1-q}\sum _{|J|=q}{\frac {1}{1-P_{J}}},}$
where m denotes the number of coupons to be collected and PJ denotes the probability of getting any coupon in the set of coupons J.

## Martingales

This section is based on.[3]

Define a discrete random process ${\displaystyle N(0),N(1),\dots }$ by letting ${\displaystyle N(t)}$ be the number of coupons not yet seen after ${\displaystyle t}$ draws. The random process is just a sequence generated by a Markov chain with states ${\displaystyle n,n-1,\dots ,1,0}$, and transition probabilities

${\displaystyle p_{i\to i-1}=i/n,\quad p_{i\to i}=1-i/n}$
Now define
${\displaystyle M(t):=N(t)\left({\frac {n}{n-1}}\right)^{t}}$
then it is a martingale, since
${\displaystyle E[M(t+1)|M(t)]=(n/(n-1))^{t+1}E[N(t+1)|N(t)]=(n/(n-1))^{t+1}(N(t)-N(t)/n)=M(t)}$
Consequently, we have ${\displaystyle E[N(t)]=n(1-1/n)^{t}}$. In particular, we have a limit law ${\displaystyle \lim _{n\to \infty }E[N(n\ln n+cn)]=e^{-c}}$ for any ${\displaystyle c>0}$. This suggests to us a limit law for ${\displaystyle T}$.

More generally, each ${\displaystyle \left({\frac {n}{n-k}}\right)^{t}N(t)\cdots (N(t)-k+1)}$ is a martingale process, which allows us to calculate all moments of ${\displaystyle N(t)}$. For example,

${\displaystyle E[N(t)^{2}]=n(n-1)\left({\frac {n-2}{n}}\right)^{t}+n\left({\frac {n-1}{n}}\right)^{t},\quad n\geq 2}$
giving another limit law ${\displaystyle \lim _{n\to \infty }Var[N(n\ln n+cn)]=e^{-c}}$. More generally,
${\displaystyle \lim _{n\to \infty }E[N(n\ln n+cn)\cdots (N(n\ln n+cn)-k+1)]=e^{-kc}}$
meaning that ${\displaystyle N(n\ln n+cn)}$ has all moments converging to constants, so it converges to some probability distribution on ${\displaystyle 0,1,2,\dots }$.

Let ${\displaystyle N}$ be the random variable with the limit distribution. We have

{\displaystyle {\begin{aligned}E[1]&=1\\E[N]&=e^{-c}\\E[N(N-1)]&=e^{-2c}\\E[N(N-1)(N-2)]&=e^{-3c}\\&\vdots \end{aligned}}}
By introducing a new variable ${\displaystyle t}$, we can sum up both sides explicitly:
${\displaystyle E[1+Nt/1!+N(N-1)t^{2}/2!+\cdots ]=1+e^{-c}t/1!+e^{-2c}t^{2}/2!+\cdots }$
giving ${\displaystyle E[(1+t)^{N}]=e^{e^{-c}t}}$.

At the ${\displaystyle t\to 0}$ limit, we have ${\displaystyle Pr(N=0)=e^{-e^{-c}}}$, which is precisely what the limit law states.

By taking the derivative ${\displaystyle d/dt}$ multiple times, we find that ${\displaystyle Pr(N=k)={\frac {e^{-kc}}{k!}}e^{-e^{-c}}}$, which is a Poisson distribution.

2. ^ E(50) = 50(1 + 1/2 + 1/3 + ... + 1/50) = 224.9603, the expected number of trials to collect all 50 coupons. The approximation ${\displaystyle n\log n+\gamma n+1/2}$ for this expected number gives in this case ${\displaystyle 50\log 50+50\gamma +1/2\approx 195.6011+28.8608+0.5\approx 224.9619}$.
1. ^ Mitzenmacher, Michael (2017). Probability and computing : randomization and probabilistic techniques in algorithms and data analysis. Eli Upfal (2nd ed.). Cambridge, United Kingdom. Theorem 5.13. ISBN 978-1-107-15488-9. OCLC 960841613.{{cite book}}: CS1 maint: location missing publisher (link)