Information matrix test explained

In econometrics, the information matrix test is used to determine whether a regression model is misspecified. The test was developed by Halbert White,^[1] who observed that in a correctly specified model and under standard regularity assumptions, the Fisher information matrix can be expressed in either of two ways: as the outer product of the gradient, or as a function of the Hessian matrix of the log-likelihood function.

Consider a linear model

y = X \beta + u , where the errors u are assumed to be distributed N ( 0 , \sigma ^{2} I ) . If the parameters \beta and \sigma ^{2} are stacked in the vector \theta ^{T} = \begin{bmatrix} \beta & \sigma ^{2} \end{bmatrix} , the resulting log-likelihood function is\ell ( \theta ) = - (n)/(2) log \sigma ^{2} - (1)/(2 \sigma ^{2}) \left( y - X \beta \right) ^{T} \left( y - X \beta \right)

The information matrix can then be expressed as

I ( \theta ) = \operatorname{E} \left[ \left( (\partial \ell ( \theta ))/(\partial \theta) \right) \left( (\partial \ell ( \theta ))/(\partial \theta) \right) ^{T} \right] that is the expected value of the outer product of the gradient or score. Second, it can be written as the negative of the Hessian matrix of the log-likelihood functionI ( \theta ) = - \operatorname{E} \left[ (\partial ^{2} \ell ( \theta ))/(\partial \theta \partial \theta ^{T}) \right]

If the model is correctly specified, both expressions should be equal. Combining the equivalent forms yields

\Delta ( \theta ) = \sum ^{n} _{i =1} \left[ (\partial ^{2} \ell( \theta ))/(\partial \theta \partial \theta ^{T}) + (\partial \ell( \theta ))/(\partial \theta) (\partial \ell ( \theta ))/(\partial \theta) \right]

where

\Delta ( \theta ) is an (r x r) random matrix, where r is the number of parameters. White showed that the elements of n ^{-1/2} \Delta ( \hat{\theta }), where \hat{\theta } is the MLE, are asymptotically normally distributed with zero means when the model is correctly specified.^[2] In small samples, however, the test generally performs poorly.^[3]

Further reading

• Book: W. . Krämer . H. . Sonnberger . The Linear Regression Model Under Test . Heidelberg . Physica-Verlag . 1986 . 3-7908-0356-1 . 105–110 .
• Book: White, Halbert . Information Matrix Testing . Estimation, Inference and Specification Analysis . New York . Cambridge University Press . 1994 . 0-521-25280-6 . 300–344 . https://books.google.com/books?id=hnNpQSf7ZlAC&pg=PA300 .

Notes and References

1. White. Halbert. Maximum Likelihood Estimation of Misspecified Models . . 1982 . 50 . 1 . 1–25 . 10.2307/1912526 . 1912526 .
2. Book: Godfrey, L. G. . Leslie G. Godfrey . Misspecification Tests in Econometrics . . 1988 . 0-521-26616-5 . 35–37 .
3. Chris . Orme . The Small-Sample Performance of the Information-Matrix Test . . 46 . 3 . 1990 . 309–331 . 10.1016/0304-4076(90)90012-I .