AutonoML: Towards an Integrated Framework for Autonomous Machine Learning (2020-12-23T00:00:00.000000Z)

TL;DR

This review seeks to motivate a more expansive perspective on what constitutes an automated/autonomous ML system, alongside consideration of how best to consolidate those elements, and develops a conceptual framework to illustrate one possible way of fusing high-level mechanisms into an autonomous ML system.

Abstract

Over the last decade, the long-running endeavour to automate high-level processes in machine learning (ML) has risen to mainstream prominence, stimulated by advances in optimisation techniques and their impact on selecting ML models/algorithms. Central to this drive is the appeal of engineering a computational system that both discovers and deploys high-performance solutions to arbitrary ML problems with minimal human interaction. Beyond this, an even loftier goal is the pursuit of autonomy, which describes the capability of the system to independently adjust an ML solution over a lifetime of changing contexts. However, these ambitions are unlikely to be achieved in a robust manner without the broader synthesis of various mechanisms and theoretical frameworks, which, at the present time, remain scattered across numerous research threads. Accordingly, this review seeks to motivate a more expansive perspective on what constitutes an automated/autonomous ML system, alongside consideration of how best to consolidate those elements. In doing so, we survey developments in the following research areas: hyperparameter optimisation, multi-component models, neural architecture search, automated feature engineering, meta-learning, multi-level ensembling, dynamic adaptation, multi-objective evaluation, resource constraints, flexible user involvement, and the principles of generalisation. We also develop a conceptual framework throughout the review, augmented by each topic, to illustrate one possible way of fusing high-level mechanisms into an autonomous ML system. Ultimately, we conclude that the notion of architectural integration deserves more discussion, without which the field of automated ML risks stifling both its technical advantages and general uptake.

Authors

Katarzyna Musial

2 papers

Bogdan Gabrys

2 papers

D. Kedziora

1 papers

TL;DR

Abstract

Authors

References415 items

Automatic recommendation of feature selection algorithms based on dataset characteristics

AlphaD3M: Machine Learning Pipeline Synthesis

Automated Machine Learning Techniques for Data Streams

Exploring Opportunistic Meta-knowledge to Reduce Search Spaces for Automated Machine Learning

PyGlove: Symbolic Programming for Automated Machine Learning

AutoWeka4MCPS-AVATAR: Accelerating Automated Machine Learning Pipeline Composition and Optimisation

Can AutoML outperform humans? An evaluation on popular OpenML datasets using AutoML Benchmark

NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size

On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice

A Review of Meta-level Learning in the Context of Multi-component, Multi-level Evolving Prediction Systems

AutoML Pipeline Selection: Efficiently Navigating the Combinatorial Space

Fast, Accurate, and Simple Models for Tabular Data via Augmented Distillation

AutoGAN-Distiller: Searching to Compress Generative Adversarial Networks

Adaptation Strategies for Automated Machine Learning on Evolving Data

AutoHAS: Differentiable Hyper-parameter and Architecture Search

AutoHAS: Efficient Hyperparameter and Architecture Search.

A Comprehensive Survey of Neural Architecture Search

Deep Multimodal Neural Architecture Search

FedNAS: Federated Deep Learning via Neural Architecture Search

AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data

Towards CRISP-ML(Q): A Machine Learning Process Model with Quality Assurance Methodology

AutoML-Zero: Evolving Machine Learning Algorithms From Scratch

RandomNet: Towards Fully Automatic Neural Architecture Design for Multimodal Learning

Federated Neural Architecture Search

AVATAR - Machine Learning Pipeline Evaluation Using Surrogate Model

Trust in AutoML: exploring information needs for establishing trust in automated machine learning systems

An Improved Online Learning Algorithm for General Fuzzy Min-Max Neural Network

Advances and Open Problems in Federated Learning

Optimizing Millions of Hyperparameters by Implicit Differentiation

Hyperparameter Importance for Image Classification by Residual Neural Networks

Human-AI Collaboration in Data Science

Towards Meta-learning of Deep Architectures for Efficient Domain Adaptation

Towards Automated Machine Learning: Evaluation and Comparison of AutoML Approaches and Tools

AutoGAN: Neural Architecture Search for Generative Adversarial Networks

AutoML: A Survey of the State-of-the-Art

Towards Automated Semi-Supervised Learning

AutoCV Challenge Design and Baseline Results

An Open Source AutoML Benchmark

Democratizing Data Science through Interactive Curation of ML Pipelines

Searching for a Robust Neural Architecture in Four GPU Hours

A Survey on Neural Architecture Search

An ADMM Based Framework for AutoML Pipeline Configuration

Benchmark and Survey of Automated Machine Learning Frameworks

NetPyNE, a tool for data-driven multiscale modeling of brain circuits

HARK Side of Deep Learning - From Grad Student Descent to Automated Machine Learning

Expert-augmented machine learning

How Far are we from Data Mining Democratisation? A Systematic Review

Towards human-guided machine learning

Evaluating the Search Phase of Neural Architecture Search

Random Search and Reproducibility for Neural Architecture Search

Democratisation of Usable Machine Learning in Computer Vision

GAMA: Genetic Automated Machine learning Assistant

The Evolved Transformer

Machine Learning and Data Mining in Pattern Recognition

Generic adaptation strategies for automated machine learning

InferLine: ML Inference Pipeline Composition Framework

Automated Algorithm Selection: Survey and Perspectives

Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead

A Very Brief and Critical Discussion on AutoML

Automated Machine Learning: From Principles to Practices

Self Hyper-Parameter Tuning for Data Streams

Preprocessor Selection for Machine Learning Pipelines

A Hybrid Approach for Automatic Model Recommendation

Hyper-parameter optimization tools comparison for multiple object tracking applications

Sherpa: Hyperparameter Optimization for Machine Learning Models

Learning to Design RNA

ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware

A Modern Take on the Bias-Variance Tradeoff in Neural Networks

ActivMetal: Algorithm Recommendation with Active Meta Learning

Automated Machine Learning Service Composition

Benchmarking Automatic Machine Learning Frameworks

Net-Net Auto Machine Learning (AutoML) Prediction of Complex Ecosystems

Northstar: An Interactive Data Science System

Towards Automated Deep Learning: Efficient Joint Neural Architecture and Hyperparameter Search

Automated Machine Learning Algorithm Mining for Classification Problem

Towards AutoML in the presence of Drift: first results