Zero-inflated model

In statistics, a zero-inflated model is a statistical model based on a zero-inflated probability distribution, i.e. a distribution that allows for frequent zero-valued observations.

Zero-inflated Poisson

The first zero-inflated model is the zero-inflated Poisson model, which concerns a random event containing excess zero-count data in unit time. For example, the number of insurance claims within a population for a certain type of risk would be zero-inflated by those people who have not taken out insurance against the risk and thus are unable to claim. The zero-inflated Poisson (ZIP) model employs two components that correspond to two zero generating processes. The first process is governed by a binary distribution that generates structural zeros. The second process is governed by a Poisson distribution that generates counts, some of which may be zero. The two model components are described as follows:

$\Pr(y_{j}=0)=\pi +(1-\pi )e^{-\lambda }$$\Pr(y_{j}=0)=\pi +(1-\pi )e^{{-\lambda }}$

$\Pr(y_{j}=h_{i})=(1-\pi ){\frac {\lambda ^{h_{i}}e^{-\lambda }}{h_{i}!}},\qquad h_{i}\geq 1$$\Pr(y_{j}=h_{i})=(1-\pi ){\frac {\lambda ^{{h_{i}}}e^{{-\lambda }}}{h_{i}!}},\qquad h_{i}\geq 1$

where the outcome variable $y_{j}$$y_{j}$ has any non-negative integer value, $\lambda _{i}$$\lambda _{i}$ is the expected Poisson count for the $i$$i$th individual; $\pi$$\pi$ is the probability of extra zeros.

The mean is $(1-\pi )\lambda$$(1-\pi )\lambda$ and the variance is $\lambda (1-\pi )(1+\lambda \pi )$$\lambda (1-\pi )(1+\lambda \pi )$.

Estimators of ZIP

The method of moments estimators are given by

${\hat {\lambda }}_{mo}={\frac {s^{2}+m^{2}-m}{m}},$${\hat {\lambda }}_{{mo}}={\frac {s^{2}+m^{2}-m}{m}},$

${\hat {\pi }}_{mo}={\frac {s^{2}-m}{s^{2}+m^{2}-m}},$${\hat {\pi }}_{{mo}}={\frac {s^{2}-m}{s^{2}+m^{2}-m}},$

where $m$$m$ is the sample mean and $s^{2}$$s^{2}$ is the sample variance.

The maximum likelihood estimator can be found by solving the following equation

${\bar {x}}(1-e^{-{\hat {\lambda }}_{ml}})={\hat {\lambda }}_{ml}\left(1-{\frac {n_{0}}{n}}\right).$${\bar {x}}(1-e^{{-{\hat {\lambda }}_{{ml}}}})={\hat {\lambda }}_{{ml}}\left(1-{\frac {n_{0}}{n}}\right).$

where ${\bar {x}}$${\bar {x}}$ is the sample mean, and ${\frac {n_{0}}{n}}$${\frac {n_{0}}{n}}$ is the observed proportion of zeros.

This can be solved by iteration, and the maximum likelihood estimator for $\pi$$\pi$ is given by

${\hat {\pi }}_{ml}=1-{\frac {\bar {x}}{{\hat {\lambda }}_{ml}}}.$${\hat {\pi }}_{{ml}}=1-{\frac {{\bar {x}}}{{\hat {\lambda }}_{{ml}}}}.$

Related models

1994, Greene considered the zero-inflated negative binomial (ZINB) model. Daniel B. Hall adapted Lambert's methodology to an upper-bounded count situation, thereby obtaining a zero-inflated binomial (ZIB) model.

Discrete pseudo compound Poisson model

If the count data $Y$$Y$ with the feature that the probability of zero is larger than the probability of nonzero, namely

$\Pr(Y=0)>0.5$$\Pr(Y=0)>0.5$

then the discrete data $Y$$Y$ obey discrete pseudo compound Poisson distribution.

In fact, let $G(z)=\sum \limits _{n=0}^{\infty }P(Y=n)z^{n}$$G(z)=\sum \limits _{n=0}^{\infty }P(Y=n)z^{n}$ be the probability generating function of $y_{i}$$y_{i}$. If $p_{0}=\Pr(Y=0)>0.5$$p_{0}=\Pr(Y=0)>0.5$, then $|G(z)|\geqslant p_{0}-\sum \limits _{i=1}^{\infty }p_{i}=2p_{0}-1>0$$|G(z)|\geqslant p_{0}-\sum \limits _{i=1}^{\infty }p_{i}=2p_{0}-1>0$. Then from Wiener–Lévy theorem, we show that $G(z)$$G(z)$ have the probability generating function of discrete pseudo compound Poisson distribution.

We say that the discrete random variable $Y$$Y$ satisfying probability generating function characterization

$G_{Y}(z)=\sum \limits _{n=0}^{\infty }P(Y=n)z^{n}=\exp \left(\sum _{k=1}^{\infty }\alpha _{k}\lambda (z^{k}-1)\right),\quad (|z|\leq 1)$$G_{Y}(z)=\sum \limits _{n=0}^{\infty }P(Y=n)z^{n}=\exp \left(\sum _{k=1}^{\infty }\alpha _{k}\lambda (z^{k}-1)\right),\quad (|z|\leq 1)$

has a discrete pseudo compound Poisson distribution with parameters

$(\lambda _{1},\lambda _{2},\ldots )=(\alpha _{1}\lambda ,\alpha _{2}\lambda ,\ldots )\in \mathbb {R} ^{\infty }\left(\sum _{k=1}^{\infty }\alpha _{k}=1,\sum \limits _{k=1}^{\infty }|\alpha _{k}|<\infty ,\alpha _{k}\in \mathbb {R} ,\lambda >0\right).$$(\lambda _{1},\lambda _{2},\ldots )=(\alpha _{1}\lambda ,\alpha _{2}\lambda ,\ldots )\in \mathbb {R} ^{\infty }\left(\sum _{k=1}^{\infty }\alpha _{k}=1,\sum \limits _{k=1}^{\infty }|\alpha _{k}|<\infty ,\alpha _{k}\in \mathbb {R} ,\lambda >0\right).$

When all the $\alpha _{k}$$\alpha _{k}$ are non-negative, it is the discrete compound Poisson distribution(non-Poisson case) with overdispersion property.

See also

• Poisson distribution
• Zero-truncated Poisson distribution
• Compound Poisson distribution
• Sparse approximation