Normal-inverse-gamma distribution explained

In probability theory and statistics, the normal-inverse-gamma distribution (or Gaussian-inverse-gamma distribution) is a four-parameter family of multivariate continuous probability distributions. It is the conjugate prior of a normal distribution with unknown mean and variance.

Definition

Suppose

x \mid \sigma ^{2,} \mu, λ \sim N (\mu,\sigma ^{2} / λ ) has a normal distribution with mean \mu and variance \sigma ^{2} / λ , where\sigma ^{2\mid\alpha,} \beta \sim \Gamma ^{-1} (\alpha,\beta) has an inverse-gamma distribution. Then (x,\sigma ^{2)} has a normal-inverse-gamma distribution, denoted as(x,\sigma ^{2)} \sim N- \Gamma ^{-1} (\mu, λ ,\alpha,\beta) .

(

NIG is also used instead of N- \Gamma ^{-1} . )

The normal-inverse-Wishart distribution is a generalization of the normal-inverse-gamma distribution that is defined over multivariate random variables.

Characterization

Probability density function

f(x,\sigma ^{2\mid\mu,λ,\alpha,\beta)} = (\sqrt{ λ)/() } \, \frac \, \left(\frac \right)^ \exp \left(-\frac \right)

For the multivariate form where

x is a k x 1 random vector,f( x ,\sigma ^{2\mid\mu,V} ^{-1} ,\alpha,\beta) = | V | ^{-1/2} {( 2 \pi) ^{-k/2} } (\beta ^{\alpha})/(\Gamma(\alpha)) \left( (1)/(\sigma ^{2}) \right) ^{\alpha} \exp \left( - (2 \beta + ( x - \boldsymbol{\mu)/()) ^{T} V ^{-1} ( x - \boldsymbol{\mu})} { 2 \sigma ^{2}} \right).

where

| V | is the determinant of the k x k matrix V . Note how this last equation reduces to the first form if k = 1 so that x , V , \boldsymbol{\mu} are scalars.

Alternative parameterization

It is also possible to let

\gamma = 1 / λ in which case the pdf becomesf(x,\sigma ^{2\mid\mu,\gamma,\alpha,\beta)} = (1)/(\sigma\sqrt{ 2 \pi\gamma) } (\beta ^{\alpha})/(\Gamma(\alpha)) \left( (1)/(\sigma ^{2}) \right) ^{\alpha} \exp \left( - (2 \gamma\beta + (x - \mu) ^{2})/(2 \gamma \sigma ^{2}) \right)

In the multivariate form, the corresponding change would be to regard the covariance matrix

V instead of its inverse V ^{-1} as a parameter.

Cumulative distribution function

F(x,\sigma ^{2\mid\mu,λ,\alpha,\beta)} = (e ^{- (\beta)/(\sigma ^{2})} \left( (\beta)/(\sigma ^{2}) \right) ^{\alpha} \left(\operatorname{erf)/(\left( (\sqrt{ λ)/((x - \mu)) )}{\sqrt{ 2 } \sigma }\right) +1 \right)}{ 2 \sigma ^{2} \Gamma (\alpha)}

Properties

Marginal distributions

Given

(x,\sigma ^{2)} \sim N- \Gamma ^{-1} (\mu, λ ,\alpha,\beta) . as above, \sigma ^{2} by itself follows an inverse gamma distribution:\sigma ^{2} \sim \Gamma ^{-1} (\alpha,\beta)

while

\sqrt{ (\alpha λ)/(\beta) } (x - \mu) follows a t distribution with 2 \alpha degrees of freedom.^[1]

In the multivariate case, the marginal distribution of

x is a multivariate t distribution:x \sim t _{2\alpha} (\boldsymbol{\mu}, (\beta)/(\alpha) V )

Scaling

Suppose

(x,\sigma ^{2)} \sim N- \Gamma ^{-1} (\mu, λ ,\alpha,\beta) .

Then for

c> 0 , (cx,c\sigma ^{2)} \sim N- \Gamma ^{-1} (c\mu, λ /c,\alpha,c\beta) .

Proof: To prove this let

(x,\sigma ^{2)} \sim N- \Gamma ^{-1} (\mu, λ ,\alpha,\beta) and fix c> 0 . Defining Y = (Y _{1,Y} _{2)=(cx,c} \sigma ^{2)} , observe that the PDF of the random variable Y evaluated at (y _{1,y} _{2)} is given by 1 /c ^{2} times the PDF of a N- \Gamma ^{-1} (\mu, λ ,\alpha,\beta) random variable evaluated at (y _{1/c,y} _{2/c)} . Hence the PDF of Y evaluated at (y _{1,y} _{2)} is given by :f _{Y(y} _{1,y} ^{2 ) = (1)/(c ^{2})} (\sqrt{ λ)/() } \, \frac \, \left(\frac \right)^ \exp \left(-\frac \right) = \frac \, \frac \, \left(\frac \right)^ \exp \left(-\frac \right).\!

The right hand expression is the PDF for a

N- \Gamma ^{-1} (c\mu, λ /c,\alpha,c\beta) random variable evaluated at (y _{1,y} _{2)} , which completes the proof.

Exponential family

Normal-inverse-gamma distributions form an exponential family with natural parameters

style\theta _{1= (-λ)/(2)} , style\theta _{2=λ} \mu , style\theta _{3=\alpha} , and style\theta _{4=- \beta + (-λ \mu ^{2})/(2)} and sufficient statistics style T _{1= (x ^{2})/(\sigma ^{2})} , style T _{2= (x)/(\sigma ^{2})} , style T _{3=log} ( (1)/(\sigma ^{2}) ) , and style T _{4= (1)/(\sigma ^{2})} .

Kullback–Leibler divergence

Measures difference between two distributions.

Posterior distribution of the parameters

See the articles on normal-gamma distribution and conjugate prior.

Interpretation of the parameters

See the articles on normal-gamma distribution and conjugate prior.

Generating normal-inverse-gamma random variates

Generation of random variates is straightforward:

1. Sample

\sigma ^{2} from an inverse gamma distribution with parameters \alpha and \beta

1. Sample

x from a normal distribution with mean \mu and variance \sigma ^{2/λ}

Related distributions

• The normal-gamma distribution is the same distribution parameterized by precision rather than variance
• A generalization of this distribution which allows for a multivariate mean and a completely unknown positive-definite covariance matrix

\sigma ^{2} V (whereas in the multivariate inverse-gamma distribution the covariance matrix is regarded as known up to the scale factor \sigma ^{2} ) is the normal-inverse-Wishart distribution

See also

• Compound probability distribution

References

• Denison, David G. T.; Holmes, Christopher C.; Mallick, Bani K.; Smith, Adrian F. M. (2002) Bayesian Methods for Nonlinear Classification and Regression, Wiley.
• Koch, Karl-Rudolf (2007) Introduction to Bayesian Statistics (2nd Edition), Springer.

Notes and References

1. Book: Ramírez-Hassan, Andrés . 4.2 Conjugate prior to exponential family Introduction to Bayesian Econometrics.