Berndt–Hall–Hall–Hausman algorithm explained

The Berndt–Hall–Hall–Hausman (BHHH) algorithm is a numerical optimization algorithm similar to the Newton–Raphson algorithm, but it replaces the observed negative Hessian matrix with the outer product of the gradient. This approximation is based on the information matrix equality and therefore only valid while maximizing a likelihood function.^[1] The BHHH algorithm is named after the four originators: Ernst R. Berndt, Bronwyn Hall, Robert Hall, and Jerry Hausman.^[2]

Usage

If a nonlinear model is fitted to the data one often needs to estimate coefficients through optimization. A number of optimisation algorithms have the following general structure. Suppose that the function to be optimized is Q(β). Then the algorithms are iterative, defining a sequence of approximations, β_k given by

\beta _{k+1} = \beta _{k} -λ _{k} A _{k} (\partial Q)/(\partial \beta) (\beta _{k} ), ,

where

\beta _{k} is the parameter estimate at step k, and λ _{k} is a parameter (called step size) which partly determines the particular algorithm. For the BHHH algorithm λ_k is determined by calculations within a given iterative step, involving a line-search until a point β_k+1 is found satisfying certain criteria. In addition, for the BHHH algorithm, Q has the formQ = \sum ^{N} _{i =1} Q _{i} and A is calculated usingA _{k} = \left[\sum ^{N} _{i =1} (\partial ln Q _{i})/(\partial \beta) (\beta _{k} ) (\partial ln Q _{i})/(\partial \beta) (\beta _{k} )'\right] ^{-1} . In other cases, e.g. Newton–Raphson, A _{k} can have other forms. The BHHH algorithm has the advantage that, if certain conditions apply, convergence of the iterative procedure is guaranteed.

See also

• Davidon–Fletcher–Powell (DFP) algorithm
• Broyden–Fletcher–Goldfarb–Shanno (BFGS) algorithm

Further reading

• V. Martin, S. Hurn, and D. Harris, Econometric Modelling with Time Series, Chapter 3 'Numerical Estimation Methods'. Cambridge University Press, 2015.
• Book: Amemiya, Takeshi . Takeshi Amemiya . Advanced Econometrics . Cambridge . Harvard University Press . 1985 . 0-674-00560-0 . 137–138 . registration .
• Book: Gill, P. . W. . Murray . M. . Wright . 1981 . Practical Optimization . Harcourt Brace . London .
• Book: Gourieroux, Christian . Alain . Monfort . Statistics and Econometric Models . Gradient Methods and ML Estimation . New York . Cambridge University Press . 1995 . 0-521-40551-3 . 452–458 . https://books.google.com/books?id=gqI-pAP2JZ8C&pg=PA452 .
• Book: Harvey, A. C. . The Econometric Analysis of Time Series . Cambridge . MIT Press . 1990 . Second . 0-262-08189-X . 137–138 .

Notes and References

1. A. . Henningsen . O. . Toomet . maxLik: A package for maximum likelihood estimation in R . Computational Statistics . 2011 . 26 . 3 . 443–458 [p. 450] . 10.1007/s00180-010-0217-1 .
2. Berndt . E. . B. . Hall . R. . Hall . J. . Hausman . 1974 . Estimation and Inference in Nonlinear Structural Models . Annals of Economic and Social Measurement . 3 . 4 . 653–665 .