# g-prior

In statistics, the g-prior is an objective prior for the regression coefficients of a multiple regression. It was introduced by Arnold Zellner.[1] It is a key tool in Bayes and empirical Bayes variable selection.[2][3]

## Definition

Consider a data set ${\displaystyle (x_{1},y_{1}),\ldots ,(x_{n},y_{n})}$, where the ${\displaystyle x_{i}}$ are Euclidean vectors and the ${\displaystyle y_{i}}$ are scalars. The multiple regression model is formulated as

${\displaystyle y_{i}=x_{i}^{\top }\beta +\varepsilon _{i}.}$

where the ${\displaystyle \varepsilon _{i}}$ are random errors. Zellner's g-prior for ${\displaystyle \beta }$ is a multivariate normal distribution with covariance matrix proportional to the inverse Fisher information matrix for ${\displaystyle \beta }$.

Assume the ${\displaystyle \varepsilon _{i}}$ are iid normal with zero mean and variance ${\displaystyle \psi ^{-1}}$. Let ${\displaystyle X}$ be the matrix with ${\displaystyle i}$th row equal to ${\displaystyle x_{i}^{\top }}$. Then the g-prior for ${\displaystyle \beta }$ is the multivariate normal distribution with prior mean a hyperparameter ${\displaystyle \beta _{0}}$ and covariance matrix proportional to ${\displaystyle \psi ^{-1}(X^{\top }X)^{-1}}$, i.e.,

${\displaystyle \beta |\psi \sim {\text{MVN}}[\beta _{0},g\psi ^{-1}(X^{\top }X)^{-1}].}$

where g is a positive scalar parameter.

## Posterior distribution of ${\displaystyle \beta }$

The posterior distribution of ${\displaystyle \beta }$ is given as

${\displaystyle \beta |\psi ,x,y\sim {\text{MVN}}{\Big [}q{\hat {\beta }}+(1-q)\beta _{0},{\frac {q}{\psi }}(X^{\top }X)^{-1}{\Big ]}.}$

where ${\displaystyle q=g/(1+g)}$ and

${\displaystyle {\hat {\beta }}=(X^{\top }X)^{-1}X^{\top }y.}$

is the maximum likelihood (least squares) estimator of ${\displaystyle \beta }$. The vector of regression coefficients ${\displaystyle \beta }$ can be estimated by its posterior mean under the g-prior, i.e., as the weighted average of the maximum likelihood estimator and ${\displaystyle \beta _{0}}$,

${\displaystyle {\tilde {\beta }}=q{\hat {\beta }}+(1-q)\beta _{0}.}$

Clearly, as g →∞, the posterior mean converges to the maximum likelihood estimator.

## Selection of g

Estimation of g is slightly less straightforward than estimation of ${\displaystyle \beta }$. A variety of methods have been proposed, including Bayes and empirical Bayes estimators.[3]

## References

1. ^ Zellner, A. (1986). "On Assessing Prior Distributions and Bayesian Regression Analysis with g Prior Distributions". In Goel, P.; Zellner, A. Bayesian Inference and Decision Techniques: Essays in Honor of Bruno de Finetti. Studies in Bayesian Econometrics. 6. New York: Elsevier. pp. 233–243. ISBN 0-444-87712-6.
2. ^ George, E.; Foster, D. P. (2000). "Calibration and empirical Bayes variable selection". Biometrika. 87 (4): 731–747. doi:10.1093/biomet/87.4.731.
3. ^ a b Liang, F.; Paulo, R.; Molina, G.; Clyde, M. A.; Berger, J. O. (2008). "Mixtures of g priors for Bayesian variable selection". Journal of the American Statistical Association. 103 (481): 410–423. doi:10.1198/016214507000001337.