Generalized iterative scaling explained

In statistics, generalized iterative scaling (GIS) and improved iterative scaling (IIS) are two early algorithms used to fit log-linear models,^[1] notably multinomial logistic regression (MaxEnt) classifiers and extensions of it such as MaxEnt Markov models^[2] and conditional random fields. These algorithms have been largely surpassed by gradient-based methods such as L-BFGS^[3] and coordinate descent algorithms.^[4]

See also

• Expectation-maximization

Notes and References

1. Generalized iterative scaling for log-linear models . Darroch, J.N. . Ratcliff, D. . The Annals of Mathematical Statistics . 43 . 5 . 1470–1480 . 1972 . 10.1214/aoms/1177692379. free .
2. McCallum. Andrew. Freitag. Dayne. Pereira. Fernando. Maximum Entropy Markov Models for Information Extraction and Segmentation. Proc. ICML 2000. 2000. 591–598.
3. Robert . Malouf . 2002 . A comparison of algorithms for maximum entropy parameter estimation . Sixth Conf. on Natural Language Learning (CoNLL) . 49–55 . dead . https://web.archive.org/web/20131101205929/http://acl.ldc.upenn.edu/W/W02/W02-2018.pdf . 2013-11-01 .
4. Hsiang-Fu . Yu . Fang-Lan . Huang . Chih-Jen . Lin . 2011 . Dual coordinate descent methods for logistic regression and maximum entropy models . Machine Learning . 85 . 1–2 . 41–75 . 10.1007/s10994-010-5221-8. free .