Regularized Evolution for Image Classifier Architecture Search (2018-02-05T00:00:00.000000Z)

TL;DR

This work evolves an image classifier---AmoebaNet-A---that surpasses hand-designs for the first time and gives evidence that evolution can obtain results faster with the same hardware, especially at the earlier stages of the search.

Abstract

The effort devoted to hand-crafting neural network image classifiers has motivated the use of architecture search to discover them automatically. Although evolutionary algorithms have been repeatedly applied to neural network topologies, the image classifiers thus discovered have remained inferior to human-crafted ones. Here, we evolve an image classifier— AmoebaNet-A—that surpasses hand-designs for the first time. To do this, we modify the tournament selection evolutionary algorithm by introducing an age property to favor the younger genotypes. Matching size, AmoebaNet-A has comparable accuracy to current state-of-the-art ImageNet models discovered with more complex architecture-search methods. Scaled to larger size, AmoebaNet-A sets a new state-of-theart 83.9% top-1 / 96.6% top-5 ImageNet accuracy. In a controlled comparison against a well known reinforcement learning algorithm, we give evidence that evolution can obtain results faster with the same hardware, especially at the earlier stages of the search. This is relevant when fewer compute resources are available. Evolution is, thus, a simple method to effectively discover high-quality architectures.

Authors

Esteban Real

5 papers

Quoc V. Le

42 papers

Yanping Huang

4 papers

TL;DR

Abstract

Authors

References91 items

GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism

Analysis of DAWNBench, a Time-to-Accuracy Machine Learning Performance Benchmark

AutoAugment: Learning Augmentation Policies from Data

Faster Discovery of Neural Architectures by Searching for Paths in a Large Model

ShakeDrop regularization

PPP-Net: Platform-aware Progressive Search for Pareto-optimal Neural Architectures

Inverted Residuals and Linear Bottlenecks: Mobile Networks for Classification, Detection and Segmentation

MobileNetV2: Inverted Residuals and Linear Bottlenecks

Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning

Progressive Neural Architecture Search

Population Based Training of Neural Networks

Simple And Efficient Architecture Search for Convolutional Neural Networks

Hierarchical Representations for Efficient Architecture Search

Swish: a Self-Gated Activation Function

Deep Reinforcement Learning that Matters

Squeeze-and-Excitation Networks

Practical Block-Wise Neural Network Architecture Generation

Practical Network Blocks Design with Q-Learning

SMASH: One-Shot Model Architecture Search through HyperNetworks

Improved Regularization of Convolutional Neural Networks with Cutout

Neural Optimizer Search with Reinforcement Learning

Learning Transferable Architectures for Scalable Image Recognition

Reinforcement Learning for Architecture Search by Network Transformation

Dual Path Networks

ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices

Efficient Architecture Search by Network Transformation

Learning Deep ResNet Blocks Sequentially using Boosting Theory

Accelerating Neural Architecture Search using Performance Prediction

Shake-Shake regularization

DeepArchitect: Automatically Designing and Training Deep Architectures

MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

A genetic programming approach to designing convolutional neural network architectures

Evolution Strategies as a Scalable Alternative to Reinforcement Learning

Genetic CNN

Large-Scale Evolution of Image Classifiers

Evolving Deep Neural Networks

PathNet: Evolution Channels Gradient Descent in Super Neural Networks

Towards Automatically-Tuned Neural Networks

PolyNet: A Pursuit of Structural Diversity in Very Deep Networks

Aggregated Residual Transformations for Deep Neural Networks

Designing Neural Network Architectures using Reinforcement Learning

Neural Architecture Search with Reinforcement Learning

Learning Curve Prediction with Bayesian Neural Networks

Deep Pyramidal Residual Networks

Xception: Deep Learning with Depthwise Separable Convolutions

Densely Connected Convolutional Networks

AdaNet: Adaptive Structural Learning of Artificial Neural Networks

Learning to learn by gradient descent by gradient descent

Convolutional Neural Fabrics

Convolution by Evolution: Differentiable Pattern Producing Networks

FractalNet: Ultra-Deep Neural Networks without Residuals

Wide Residual Networks

Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning

Deep Residual Learning for Image Recognition

Efficient and Robust Automated Machine Learning

Rethinking the Inception Architecture for Computer Vision

Speeding Up Automatic Hyperparameter Optimization of Deep Neural Networks by Extrapolation of Learning Curves

Spatial Transformer Networks

Long-term recurrent convolutional networks for visual recognition and description

Going deeper with convolutions

Very Deep Convolutional Networks for Large-Scale Image Recognition

Large-Scale Video Classification with Convolutional Neural Networks

Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition

Return of the Devil in the Details: Delving Deep into Convolutional Nets

Data from Paper “False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant”

OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks

Network In Network

Regularization of Neural Networks using DropConnect

ImageNet classification with deep convolutional neural networks

Multi-column deep neural networks for image classification

Random Search for Hyper-Parameter Optimization

ImageNet: A large-scale hierarchical image database

A Hypercube-Based Encoding for Evolving Large-Scale Neural Networks

Neuroevolution: from architectures to learning

Compositional pattern producing networks: A novel abstraction of development

ALPS: the age-layered population structure for reducing the problem of premature convergence