Compound probability distribution

From Wikipedia, the free encyclopedia
Jump to: navigation, search

In probability and statistics, a compound probability distribution (also known as a mixture distribution or contagious distribution) is the probability distribution that results from assuming that a random variable is distributed according to some parametrized distribution, with (some of) the parameters of that distribution themselves being random variables.

The unconditional compound distribution is the result of marginalizing (integrating) over the latent random variable(s) representing the parameter(s) of the conditional distribution.

Definition[edit]

A compound probability distribution is the probability distribution that results from assuming that a random variable is distributed according to some parametrized distribution with an unknown parameter that is again distributed according to some other distribution . The resulting distribution is said to be the distribution that results from compounding with . The parameter's distribution is also called the mixing distribution or latent distribution. Technically, the unconditional distribution results from marginalizing over , i.e., from integrating out the unknown parameter(s) . Its probability density function is given by:

The same formula applies analogously if some or all of the variables are vectors.

From the above formula, one can see that a compound distribution essentially is a special case of a marginal distribution: The joint distribution of and is given by , and the compound results as its marginal distribution: . If the domain of is discrete, then the distribution is again a special case of a mixture distribution.

Properties[edit]

A compound distribution resembles in many ways the original distribution that generated it, but typically has greater variance, and often heavy tails as well. The support of is the same as the support of the , and often the shape is broadly similar as well. The parameters of include any parameters of or that have not been marginalized out.

The compound distribution's first two moments are given by

and

.

Applications[edit]

Testing[edit]

Distributions of common test statistics result as compound distributions under their null hypothesis, for example in Student's t-test (where the test statistic results as the ratio of a normal and a chi-squared random variable), or in the F-test (where the test statistic is the ratio of two chi-squared random variables).

Overdispersion modeling[edit]

Compound distributions are useful for modeling outcomes exhibiting overdispersion, i.e., a greater amount of variability than would be expected under a certain model. For example, count data are commonly modeled using the Poisson distribution, whose variance is equal to its mean. The distribution may be generalized by allowing for variability in its rate parameter, implemented via a gamma distribution, which results in a marginal negative binomial distribution. This distribution is similar in its shape to the Poisson distribution, but it allows for larger variances. Similarly, a binomial distribution may be generalized to allow for additional variability by compounding it with a beta distribution for its success probability parameter, which results in a beta-binomial distribution.

Bayesian inference[edit]

Besides ubiquitous marginal distributions that may be seen as special cases of compound distributions, in Bayesian inference, compound distributions arise when, in the notation above, F represents the distribution of future observations and G is the posterior distribution of the parameters of F, given the information in a set of observed data. This gives a posterior predictive distribution. Correspondingly, for the prior predictive distribution, F is the distribution of a new data point while G is the prior distribution of the parameters.

Convolution[edit]

Convolution of probability distributions (to derive the probability distribution of sums of random variables) may also be seen as a special case of compounding; here the sum's distribution essentially results from considering one summand as a random location parameter for the other summand.[1]

Computation[edit]

Compound distributions derived from exponential family distributions often have a closed form. If analytical integration is not possible, numerical methods may be necessary.

Compound distributions may relatively easily be investigated using Monte Carlo methods, i.e., by generating random samples. It is often easy to generate random numbers from the distributions as well as and then utilize these to perform collapsed Gibbs sampling to generate samples from .

A compound distribution may usually also be approximated to a sufficient degree by a mixture distribution using a finite number of mixture components, allowing to derive approximate density, distribution function etc.[1]

Parameter estimation (maximum-likelihood or maximum-a-posteriori estimation) within a compound distribution model may sometimes be simplified by utilizing the EM-algorithm.[2]

Examples[edit]

See also[edit]

References[edit]

  1. ^ a b Röver, C.; Friede, T. (2017). "Discrete approximation of a mixture distribution via restricted divergence". Journal of Computational and Graphical Statistics. 26 (1): 217–222. arXiv:1602.04060Freely accessible. doi:10.1080/10618600.2016.1276840. 
  2. ^ Gelman, A.; Carlin, J. B.; Stern, H.; Rubin, D. B. (1997). "9.5 Finding marginal posterior modes using EM and related algorithms". Bayesian Data Analysis (1st ed.). Boca Raton: Chapman & Hall / CRC. p. 276. 
  3. ^ Mood, A. M.; Graybill, F. A.; Boes, D. C. (1974). Introduction to the theory of statistics (3rd ed.). New York: McGraw-Hill. 
  4. ^ Johnson, N. L.; Kemp, A. W.; Kotz, S. (2005). "6.2.2". Univariate discrete distributions (3rd ed.). New York: Wiley. p. 253. 
  5. ^ Gelman, A.; Carlin, J. B.; Stern, H.; Dunson, D. B.; Vehtari, A.; Rubin, D. B. (2014). Bayesian Data Analysis (3rd ed.). Boca Raton: Chapman & Hall / CRC. 
  6. ^ Lawless, J.F. (1987). "Negative binomial and mixed Poisson regression". The Canadian Journal of Statistics. 15 (3): 209–225. doi:10.2307/3314912. 
  7. ^ Teich, M. C.; Diament, P. (1989). "Multiply stochastic representations for K distributions and their Poisson transforms". Journal of the Optical Society of America A: Optics, Image Science and Vision. 6 (1): 80–91. doi:10.1364/JOSAA.6.000080. 
  8. ^ Johnson, N. L.; Kotz, S.; Balakrishnan, N. (1994). "20 Pareto distributions". Continuous univariate distributions. 1 (2nd ed.). New York: Wiley. p. 573. 
  9. ^ Dubey, S. D. (1970). "Compound gamma, beta and F distributions". Metrika. 16: 27–31. doi:10.1007/BF02613934. 

Further reading[edit]

  • Lindsay, B. G. (1995), Mixture models: theory, geometry and applications, NSF-CBMS Regional Conference Series in Probability and Statistics, 5, Hayward, CA, USA: Institute of Mathematical Statistics, ISBN 0-940600-32-3 
  • Seidel, W. (2010), "Mixture models", in Lovric, M., International Encyclopedia of Statistical Science, Heidelberg: Springer, pp. 827–829, ISBN 978-3-642-04898-2, doi:10.1007/978-3-642-04898-2 
  • Mood, A. M.; Graybill, F. A.; Boes, D. C. (1974), "III.4.3 Contagious distributions and truncated distributions", Introduction to the theory of statistics (3rd ed.), New York: McGraw-Hill, ISBN 0-07-042864-6 
  • Johnson, N. L.; Kemp, A. W.; Kotz, S. (2005), "8 Mixture distributions", Univariate discrete distributions, New York: Wiley, ISBN 978-0-471-27246-5