Normal-inverse-gamma distribution

From Wikipedia, the free encyclopedia
Jump to: navigation, search
normal-inverse-gamma
Parameters location (real)
(real)
(real)
(real)
Support
PDF

In probability theory and statistics, the normal-inverse-gamma distribution (or Gaussian-inverse-gamma distribution) is a four-parameter family of multivariate continuous probability distributions. It is the conjugate prior of a normal distribution with unknown mean and variance.

Definition[edit]

Suppose

has a normal distribution with mean and variance , where

has an inverse gamma distribution. Then has a normal-inverse-gamma distribution, denoted as

( is also used instead of )

In a multivariate form of the normal-inverse-gamma distribution, -- that is, conditional on , is a random vector that follows the multivariate normal distribution with mean and covariance -- while, as in the univariate case, .

Characterization[edit]

Probability density function[edit]

For the multivariate form where is a random vector,

where is the determinant of the matrix . Note how this last equation reduces to the first form if so that are scalars.

Alternative parameterization[edit]

It is also possible to let in which case the pdf becomes

In the multivariate form, the corresponding change would be to regard the covariance matrix instead of its inverse as a parameter.

Cumulative distribution function[edit]

Differential equation[edit]

The probability density function of the normal-inverse-gamma distribution is a solution to the following differential equation:

Properties[edit]

Marginal distributions[edit]

Given as above, by itself follows an inverse gamma distribution:

while follows a t distribution with degrees of freedom.

In the multivariate case, the marginal distribution of is a multivariate t distribution:

Summation[edit]

Scaling[edit]

Exponential family[edit]

Information entropy[edit]

Kullback-Leibler divergence[edit]

Maximum likelihood estimation[edit]

Posterior distribution of the parameters[edit]

See the articles on normal-gamma distribution and conjugate prior.

Interpretation of the parameters[edit]

See the articles on normal-gamma distribution and conjugate prior.

Generating normal-inverse-gamma random variates[edit]

Generation of random variates is straightforward:

  1. Sample from an inverse gamma distribution with parameters and
  2. Sample from a normal distribution with mean and variance

Related distributions[edit]

  • The normal-gamma distribution is the same distribution parameterized by precision rather than variance
  • A generalization of this distribution which allows for a multivariate mean and a completely unknown positive-definite covariance matrix (whereas in the multivariate inverse-gamma distribution the covariance matrix is regarded as known up to the scale factor ) is the normal-inverse-Wishart distribution

References[edit]

  • Denison, David G. T. ; Holmes, Christopher C.; Mallick, Bani K.; Smith, Adrian F. M. (2002) Bayesian Methods for Nonlinear Classification and Regression, Wiley. ISBN 0471490369
  • Koch, Karl-Rudolf (2007) Introduction to Bayesian Statistics (2nd Edition), Springer. ISBN 354072723X