3260 papers • 126 benchmarks • 313 datasets
Automated Machine Learning (AutoML) is a general concept which covers diverse techniques for automated model learning including automatic data preprocessing, architecture search, and model selection. Source: Evaluating recommender systems for AI-driven data science (1905.09205) Source: CHOPT : Automated Hyperparameter Optimization Framework for Cloud-Based Machine Learning Platforms
(Image credit: Papersgraph)
These leaderboards are used to track progress in automl-4
Use these libraries to find automl-4 models and implementations
This paper systematically study neural network architecture design choices for object detection and proposes a weighted bi-directional feature pyramid network (BiFPN) and a compound scaling method that uniformly scales the resolution, depth, and width for all backbone, feature network, and box/class prediction networks at the same time.
This paper introduces EfficientNetV2, a new family of convolutional networks that have faster training speed and better parameter efficiency than previous models that were searched from the search space enriched with new ops such as Fused-MBConv.
A novel framework enabling Bayesian optimization to guide the network morphism for efficient neural architecture search is proposed and an open-source AutoML system based on the developed framework is built, namely Auto-Keras.
This paper proposes a new mixed depthwise convolution (MixConv), which naturally mixes up multiple kernel sizes in a single convolution, and improves the accuracy and efficiency for existing MobileNets on both ImageNet classification and COCO object detection.
This paper proposes AutoML for Model Compression (AMC) which leverages reinforcement learning to efficiently sample the design space and can improve the model compression quality and achieves state-of-the-art model compression results in a fully automated way without any human efforts.
The Machine Learning Bazaar is introduced, a new framework for developing machine learning and automated machine learning software systems that provides solutions to a variety of data modalities and problem types and pair these pipelines with a hierarchy of AutoML strategies - Bayesian optimization and bandit learning.
This work introduces AutoGluon-Tabular, an open-source AutoML framework that requires only a single line of Python to train highly accurate machine learning models on an unprocessed tabular dataset such as a CSV file.
We present TabPFN, a trained Transformer that can do supervised classification for small tabular datasets in less than a second, needs no hyperparameter tuning and is competitive with state-of-the-art classification methods. TabPFN performs in-context learning (ICL), it learns to make predictions using sequences of labeled examples (x, f(x)) given in the input, without requiring further parameter updates. TabPFN is fully entailed in the weights of our network, which accepts training and test samples as a set-valued input and yields predictions for the entire test set in a single forward pass. TabPFN is a Prior-Data Fitted Network (PFN) and is trained offline once, to approximate Bayesian inference on synthetic datasets drawn from our prior. This prior incorporates ideas from causal reasoning: It entails a large space of structural causal models with a preference for simple structures. On the 18 datasets in the OpenML-CC18 suite that contain up to 1 000 training data points, up to 100 purely numerical features without missing values, and up to 10 classes, we show that our method clearly outperforms boosted trees and performs on par with complex state-of-the-art AutoML systems with up to 230$\times$ speedup. This increases to a 5 700$\times$ speedup when using a GPU. We also validate these results on an additional 67 small numerical datasets from OpenML. We provide all our code, the trained TabPFN, an interactive browser demo and a Colab notebook at https://github.com/automl/TabPFN.
Fidelity Ensemble Surrogate (MFES) is proposed, an efficient Hyperband method that is capable of utilizing both the high-fidelity and low-f fidelity measurements to accelerate the convergence of HPO tasks.
Adding a benchmark result helps the community track progress.