Normal-inverse-Wishart distribution

From Wikipedia, the free encyclopedia
Jump to: navigation, search
normal-inverse-Wishart
Notation (\boldsymbol\mu,\boldsymbol\Sigma) \sim \mathrm{NIW}(\boldsymbol\mu_0,\lambda,\boldsymbol\Psi,\nu)
Parameters \boldsymbol\mu_0\in\mathbb{R}^D\, location (vector of real)
\lambda > 0\, (real)
\boldsymbol\Psi \in\mathbb{R}^{D\times D} inverse scale matrix (pos. def.)
\nu > D-1\, (real)
Support \boldsymbol\mu\in\mathbb{R}^D ; \boldsymbol\Sigma \in\mathbb{R}^{D\times D} covariance matrix (pos. def.)
pdf f(\boldsymbol\mu,\boldsymbol\Sigma|\boldsymbol\mu_0,\lambda,\boldsymbol\Psi,\nu) = \mathcal{N}(\boldsymbol\mu|\boldsymbol\mu_0,\tfrac{1}{\lambda}\boldsymbol\Sigma)\ \mathcal{W}^{-1}(\boldsymbol\Sigma|\boldsymbol\Psi,\nu)

In probability theory and statistics, the normal-inverse-Wishart distribution (or Gaussian-inverse-Wishart distribution) is a multivariate four-parameter family of continuous probability distributions. It is the conjugate prior of a multivariate normal distribution with unknown mean and covariance matrix (the inverse of the precision matrix).[1]

Definition[edit]

Suppose

  \boldsymbol\mu|\boldsymbol\mu_0,\lambda,\boldsymbol\Sigma \sim \mathcal{N}\left(\boldsymbol\mu\Big|\boldsymbol\mu_0,\frac{1}{\lambda}\boldsymbol\Sigma\right)

has a multivariate normal distribution with mean \boldsymbol\mu_0 and covariance matrix \tfrac{1}{\lambda}\boldsymbol\Sigma, where

\boldsymbol\Sigma|\boldsymbol\Psi,\nu \sim \mathcal{W}^{-1}(\boldsymbol\Sigma|\boldsymbol\Psi,\nu)

has an inverse Wishart distribution. Then (\boldsymbol\mu,\boldsymbol\Sigma) has a normal-inverse-Wishart distribution, denoted as

 (\boldsymbol\mu,\boldsymbol\Sigma) \sim \mathrm{NIW}(\boldsymbol\mu_0,\lambda,\boldsymbol\Psi,\nu)  .

Characterization[edit]

Probability density function[edit]

f(\boldsymbol\mu,\boldsymbol\Sigma|\boldsymbol\mu_0,\lambda,\boldsymbol\Psi,\nu) = \mathcal{N}\left(\boldsymbol\mu\Big|\boldsymbol\mu_0,\frac{1}{\lambda}\boldsymbol\Sigma\right) \mathcal{W}^{-1}(\boldsymbol\Sigma|\boldsymbol\Psi,\nu)

Properties[edit]

Scaling[edit]

Marginal distributions[edit]

By construction, the marginal distribution over \boldsymbol\Sigma is an inverse Wishart distribution, and the conditional distribution over \boldsymbol\mu given \boldsymbol\Sigma is a multivariate normal distribution. The marginal distribution over \boldsymbol\mu is a multivariate t-distribution.

Posterior distribution of the parameters[edit]

Generating normal-inverse-Wishart random variates[edit]

Generation of random variates is straightforward:

  1. Sample \boldsymbol\Sigma from an inverse Wishart distribution with parameters \boldsymbol\Psi and \nu
  2. Sample \boldsymbol\mu from a multivariate normal distribution with mean \boldsymbol\mu_0 and variance \boldsymbol \tfrac{1}{\lambda} \boldsymbol\Sigma

Related distributions[edit]

Notes[edit]

  1. ^ Murphy, Kevin P. (2007). "Conjugate Bayesian analysis of the Gaussian distribution." [1]

References[edit]

  • Bishop, Christopher M. (2006). Pattern Recognition and Machine Learning. Springer Science+Business Media.
  • Murphy, Kevin P. (2007). "Conjugate Bayesian analysis of the Gaussian distribution." [2]