LogitBoost explained

In machine learning and computational learning theory, LogitBoost is a boosting algorithm formulated by Jerome Friedman, Trevor Hastie, and Robert Tibshirani.

The original paper casts the AdaBoost algorithm into a statistical framework.^[1] Specifically, if one considers AdaBoost as a generalized additive model and then applies the cost function of logistic regression, one can derive the LogitBoost algorithm.^[2]

Minimizing the LogitBoost cost function

LogitBoost can be seen as a convex optimization. Specifically, given that we seek an additive model of the form

f = \sum _{t} \alpha _{t} h _{t}

the LogitBoost algorithm minimizes the logistic loss:

\sum _{i} log \left( 1 + e ^{- y _{i} f(x _{i)}} \right)

See also

• Gradient boosting
• Logistic model tree

Notes and References

1. Jerome . Friedman . Trevor . Hastie . Robert . Tibshirani . Additive logistic regression: a statistical view of boosting . Annals of Statistics . 28 . 2 . 2000 . 337–407 . 10.1.1.51.9525 . 10.1214/aos/1016218223.
2. Web site: Machine Learning Algorithms for Beginners . 2023-10-01 . en-US.