G-prior explained

In statistics, the g-prior is an objective prior for the regression coefficients of a multiple regression. It was introduced by Arnold Zellner.^[1] It is a key tool in Bayes and empirical Bayes variable selection.^{[2] [3]}

Definition

Consider a data set

(x _{1,y} _{1),\ldots,(x} _{n,y} _{n)} , where the x _{i} are Euclidean vectors and the y _{i} are scalars.The multiple regression model is formulated asy _{i} = x ^{\top\beta} _{i} + \varepsilon _{i.} where the \varepsilon _{i} are random errors.Zellner's g-prior for \beta is a multivariate normal distribution with covariance matrix proportional to the inverse Fisher information matrix for \beta , similar to a Jeffreys prior.

Assume the

\varepsilon _{i} are i.i.d. normal with zero mean and variance \psi ^{-1} . Let X be the matrix with i th row equal to x ^{\top} _{i} .Then the g-prior for \beta is the multivariate normal distribution with prior mean a hyperparameter \beta _{0} and covariance matrix proportional to \psi ^{-1} (X ^{\top} X) ^{-1} , i.e.,\beta |\psi \sim N [\beta ^{-1} _{0 ,g\psi} (X ^{\top} X) ^{-1} ]. where g is a positive scalar parameter.

Posterior distribution of beta

The posterior distribution of

\beta is given as\beta |\psi,x,y \sim N [ q\hat\beta + ( 1- q)\beta _{0 , (q\psi(X)/(^{\top})} X) ^{-1} ] . where q = g/( 1+ g) and\hat\beta = (X ^{\top} X) ^{-1} X ^{\top} y. is the maximum likelihood (least squares) estimator of \beta . The vector of regression coefficients \beta can be estimated by its posterior mean under the g-prior, i.e., as the weighted average of the maximum likelihood estimator and \beta _{0} ,\tilde\beta = q\hat\beta + ( 1- q)\beta _{0.} Clearly, as g →∞, the posterior mean converges to the maximum likelihood estimator.

Selection of g

Estimation of g is slightly less straightforward than estimation of

\beta .A variety of methods have been proposed, including Bayes and empirical Bayes estimators.

Further reading

• Book: Datta, Jyotishka . Jayanta K. . Ghosh . 2015 . In Search of Optimal Objective Priors for Model Selection and Estimation . 225–243 . Current Trends in Bayesian Methodology with Applications . Satyanshu Kumar . Upadhyay . Umesh . Singh . Dipak K. . Dey . Appaia . Loganathan . 1 . CRC Press . 978-1-4822-3511-1 .
• Book: Marin, Jean-Michel . Christian P. . Robert . Regression and Variable Selection . 47–84 . Bayesian Core : A Practical Approach to Computational Bayesian Statistics . New York . Springer . 2007 . 978-0-387-38979-0 . 10.1007/978-0-387-38983-7_3 .

Notes and References

1. Book: Zellner, A. . 1986 . On Assessing Prior Distributions and Bayesian Regression Analysis with g Prior Distributions . Bayesian Inference and Decision Techniques: Essays in Honor of Bruno de Finetti . P. . Goel . A. . Zellner . Studies in Bayesian Econometrics and Statistics . 6 . New York . Elsevier . 233–243 . 978-0-444-87712-3 .
2. George . E. . Foster . D. P. . 2000 . Calibration and empirical Bayes variable selection . . 87 . 4 . 731–747 . 10.1093/biomet/87.4.731 . 10.1.1.18.3731 .
3. Liang . F. . Paulo . R. . Molina . G. . Clyde . M. A. . Berger . J. O. . 2008 . Mixtures of g priors for Bayesian variable selection . . 103 . 481 . 410–423 . 10.1198/016214507000001337 . 10.1.1.206.235 .