Normal-inverse-gamma distribution

From Wikipedia, the free encyclopedia
Jump to navigation Jump to search
normal-inverse-gamma
Probability density function
Probability density function of normal-inverse-gamma distribution for α = 1.0, 2.0 and 4.0, plotted in shifted and scaled coordinates.
Parameters location (real)
(real)
(real)
(real)
Support
PDF
Mean


, for
Mode


Variance

, for
, for

, for

In probability theory and statistics, the normal-inverse-gamma distribution (or Gaussian-inverse-gamma distribution) is a four-parameter family of multivariate continuous probability distributions. It is the conjugate prior of a normal distribution with unknown mean and variance.

Definition[edit]

Suppose

has a normal distribution with mean and variance , where

has an inverse gamma distribution. Then has a normal-inverse-gamma distribution, denoted as

( is also used instead of )

The normal-inverse-Wishart distribution is a generalization of the normal-inverse-gamma distribution that is defined over multivariate random variables.

Characterization[edit]

Probability density function[edit]

For the multivariate form where is a random vector,

where is the determinant of the matrix . Note how this last equation reduces to the first form if so that are scalars.

Alternative parameterization[edit]

It is also possible to let in which case the pdf becomes

In the multivariate form, the corresponding change would be to regard the covariance matrix instead of its inverse as a parameter.

Cumulative distribution function[edit]

Properties[edit]

Marginal distributions[edit]

Given as above, by itself follows an inverse gamma distribution:

while follows a t distribution with degrees of freedom. [1]

Proof for

For probability density function is

Marginal distribution over is

Except for normalization factor, expression under the integral coincides with Inverse-gamma distribution

with , , .

Since , and

Substituting this expression and factoring dependence on ,

Shape of generalized Student's t-distribution is

.

Marginal distribution follows t-distribution with degrees of freedom

.

In the multivariate case, the marginal distribution of is a multivariate t distribution:

Summation[edit]

Scaling[edit]

Suppose

Then for ,


Proof: To prove this let and fix . Defining , observe that the PDF of the random variable evaluated at is given by times the PDF of a random variable evaluated at . Hence the PDF of evaluated at is given by :


The right hand expression is the PDF for a random variable evaluated at , which completes the proof.

Exponential family[edit]

Normal distributions form an exponential family with natural parameters , , , and and sufficient statistics , , , and .

Information entropy[edit]

Kullback–Leibler divergence[edit]

Measures difference between two distributions.

Maximum likelihood estimation[edit]

Posterior distribution of the parameters[edit]

See the articles on normal-gamma distribution and conjugate prior.

Interpretation of the parameters[edit]

See the articles on normal-gamma distribution and conjugate prior.

Generating normal-inverse-gamma random variates[edit]

Generation of random variates is straightforward:

  1. Sample from an inverse gamma distribution with parameters and
  2. Sample from a normal distribution with mean and variance

Related distributions[edit]

  • The normal-gamma distribution is the same distribution parameterized by precision rather than variance
  • A generalization of this distribution which allows for a multivariate mean and a completely unknown positive-definite covariance matrix (whereas in the multivariate inverse-gamma distribution the covariance matrix is regarded as known up to the scale factor ) is the normal-inverse-Wishart distribution

See also[edit]

References[edit]

  1. ^ Murphy, Kevin P (2007). "Conjugate Bayesian analysis of the Gaussian distribution" (PDF). Retrieved 4 October 2021.{{cite web}}: CS1 maint: url-status (link)
  • Denison, David G. T. ; Holmes, Christopher C.; Mallick, Bani K.; Smith, Adrian F. M. (2002) Bayesian Methods for Nonlinear Classification and Regression, Wiley. ISBN 0471490369
  • Koch, Karl-Rudolf (2007) Introduction to Bayesian Statistics (2nd Edition), Springer. ISBN 354072723X