Auto-WEKA explained

Auto-WEKA is an automated machine learning system based on WEKA by Chris Thornton, Frank Hutter, Holger H. Hoos and Kevin Leyton-Brown.^[1] An extended version was published as Auto-WEKA 2.0.^[2] Auto-WEKA was named the first prominent AutoML system in a neutral comparison study.^[3]

It received the test-of-time award of the SIGKDD conference in 2023.^[4]

Description

Auto-WEKA introduced the Combined Algorithm Selection and Hyperparameter optimization (CASH) problem, that extends both the Algorithm selection problem and the Hyperparameter optimization problem, by searching for the best algorithm and also its hyperparameters for a given dataset. Baratchi et al. state that "[T]he real power of AutoML was unlocked through the definition of the combined algorithm selection and hyperparameter optimisation problem".^[5]

The CASH for formalism was picked up and also extended by later AutoML systems and methods such as Auto-sklearn^[6], ATM^[7], AutoPrognosis^[8], MCPS^[9], MOSAIC^[10], naive AutoML^[11] and ADMM^[12] .

Notes and References

1. Web site: Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms. Chris. Thornton. Frank. Hutter. Holger H.. Hoos. Kevin. Leyton-Brown. August 11, 2013. Association for Computing Machinery. 847–855. ACM Digital Library. 10.1145/2487575.2487629.
2. Auto-WEKA 2.0: Automatic model selection and hyperparameter optimization in WEKA. Lars. Kotthoff. Chris. Thornton. Holger H.. Hoos. Frank. Hutter. Kevin. Leyton-Brown. August 12, 2017. Journal of Machine Learning Research. 18. 25. 1–5. jmlr.org.
3. Gijsbers . Pieter . Bueno . Marcos L. P. . AMLB: an AutoML Benchmark . Journal of Machine Learning Research . 2024 . 25 . 6 . 2207.12560 .
4. Web site: KDD 2023 - Awards. kdd.org.
5. Baratchi . Mitra . Wang . Can . Limmer . Steffen . van Rijn . Jan N. . Hoos . Holger . Bäck . Thomas . Olhofer . Thomas . Automated machine learning: past, present and future . Artificial Intelligence Review . 57 . 5 . 2 . 10.1007/s10462-024-10726-1 . 2024. free .
6. Feurer . Matthias . Klein . Aaron . Eggensperger . Katharina . Springenberg . Jost Tobias . Blum . Manuel . Hutter . Frank . Efficient and Robust Automated Machine Learning . Advances in Neural Information Processing Systems . 28 . 2015 .
7. Swearingen . Thomas . Drevo . Will . Cyphers . Benett . Cuesta-Infante . Alfredo . Ross . Arun . Veeramachaneni . Kalyan . ATM: A distributed, collaborative, scalable system for automated machine learning . 2017 IEEE International Conference on Big Data (Big Data). 10.1109/BigData.2017.8257923 . 2017.
8. Alaa . Ahmed M. . van der Schaar . Mihaela . AutoPrognosis: Automated Clinical Prognostic Modeling via Bayesian Optimization with Structured Kernel Learning . Proceedings of the 35th International Conference on Machine Learning . 2018 .
9. Salvador . Manuel Martin . Budka . Marcin . Gabrys . Bogdan . Automatic Composition and Optimization of Multicomponent Predictive Systems With an Extended Auto-WEKA . IEEE Transactions on Automation Science and Engineering . 2019 . 16 . 2 . 946–959 . 10.1109/TASE.2018.2876430 . 1612.08789 .
10. Rakotoarison . Herilalaina . Schoenauer . Marc . Sebag . Michèle . Automated Machine Learning with Monte-Carlo Tree Search . Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence . 2019 . 10.24963/ijcai.2019/457 . 1906.00170 .
11. Mohr . Felix . Wever . Marcel . Naive automated machine learning . Machine Learning . 2023 . 112 . 4 . 1131–1170 . 10.1007/s10994-022-06200-0 . free .
12. Liu . Sijia . Ram . Parikshit . Vijaykeerthy . Deepak . Bouneffouf . Djallel . Bramble . Gregory . Samulowitz . Horst . Wang . Dakuo . Conn . Andrew . Gray . Alexander . An ADMM based framework for automl pipeline configuration . Proceedings of the AAAI Conference on Artificial Intelligence . 34 . 4 . 10.1609/aaai.v34i04.5926 . 2020 . 1905.00424 .