Benchmarking Automatic Machine Learning Frameworks
Published in
arXiv.org(2018)External Links:
TL
TL;DR
This work tests auto-sklearn, TPOT, auto_ml, and H2O's AutoML solution against a compiled set of regression and classification datasets sourced from OpenML and finds that auto-Sklearn performs the best across classification datasets and TPOT performs thebest across regression datasets.
Abstract
AutoML serves as the bridge between varying levels of expertise when designing machine learning systems and expedites the data science process. A wide range of techniques is taken to address this, however there does not exist an objective comparison of these techniques. We present a benchmark of current open source AutoML solutions using open source datasets. We test auto-sklearn, TPOT, auto_ml, and H2O's AutoML solution against a compiled set of regression and classification datasets sourced from OpenML and find that auto-sklearn performs the best across classification datasets and TPOT performs the best across regression datasets.
References5 items
1
OpenML Benchmarking Suites and the OpenML100
2
Automating Biomedical Data Science Through Tree-Based Pipeline Optimization
3
Efficient and Robust Automated Machine Learning
4
An updated performance comparison of virtual machines and Linux containers
5