Neural Architecture Search with Reinforcement Learning (2016-11-04T00:00:00.000000Z)

TL;DR

This paper uses a recurrent network to generate the model descriptions of neural networks and trains this RNN with reinforcement learning to maximize the expected accuracy of the generated architectures on a validation set.

Abstract

Neural networks are powerful and flexible models that work well for many difficult learning tasks in image, speech and natural language understanding. Despite their success, neural networks are still hard to design. In this paper, we use a recurrent network to generate the model descriptions of neural networks and train this RNN with reinforcement learning to maximize the expected accuracy of the generated architectures on a validation set. On the CIFAR-10 dataset, our method, starting from scratch, can design a novel network architecture that rivals the best human-invented architecture in terms of test set accuracy. Our CIFAR-10 model achieves a test error rate of 3.65, which is 0.09 percent better and 1.05x faster than the previous state-of-the-art model that used a similar architectural scheme. On the Penn Treebank dataset, our model can compose a novel recurrent cell that outperforms the widely-used LSTM cell, and other state-of-the-art baselines. Our cell achieves a test set perplexity of 62.4 on the Penn Treebank, which is 3.6 perplexity better than the previous state-of-the-art model. The cell can also be transferred to the character language modeling task on PTB and achieves a state-of-the-art perplexity of 1.214.

Authors

Quoc V. Le

42 papers

Barret Zoph

12 papers

TL;DR

Abstract

Authors

References73 items

Towards Automatically-Tuned Neural Networks

Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling

HyperNetworks

Pointer Sentinel Mixture Models

Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation

Densely Connected Convolutional Networks

Using the Output Embedding to Improve Language Models

Recurrent Highway Networks

Learning to learn by gradient descent by gradient descent

Convolutional Neural Fabrics

Learning to Optimize

FractalNet: Ultra-Deep Neural Networks without Residuals

Wide Residual Networks

Deep Networks with Stochastic Depth

Identity Mappings in Deep Residual Networks

Learning to Compose Neural Networks for Question Answering

A Theoretically Grounded Application of Dropout in Recurrent Neural Networks

Human-level concept learning through probabilistic program induction

Deep Residual Learning for Image Recognition

Minimum Risk Training for Neural Machine Translation

Sequence Level Training with Recurrent Neural Networks

Neural Programmer-Interpreters

Neural Programmer: Inducing Latent Programs with Gradient Descent

Character-Aware Neural Language Models

An Empirical Exploration of Recurrent Network Architectures

Pointer Networks

Highway Networks

Scalable Bayesian Optimization Using Deep Neural Networks

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Adam: A Method for Stochastic Optimization

Striving for Simplicity: The All Convolutional Net

Example Selection For Dictionary Learning

Deeply-Supervised Nets

Going deeper with convolutions

Sequence to Sequence Learning with Neural Networks

Recurrent Neural Network Regularization

Very Deep Convolutional Networks for Large-Scale Image Recognition

Neural Machine Translation by Jointly Learning to Align and Translate

How to Construct Deep Recurrent Neural Networks

Network In Network

On the importance of initialization and momentum in deep learning

Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures

ImageNet classification with deep convolutional neural networks

Large Scale Distributed Deep Networks

Context dependent recurrent neural network language model

Practical Bayesian Optimization of Machine Learning Algorithms

Random Search for Hyper-Parameter Optimization

Algorithms for Hyper-Parameter Optimization

Rectified Linear Units Improve Restricted Boltzmann Machines

Learning Programs: A Hierarchical Bayesian Approach

What is the best multi-stage architecture for object recognition?

A Hypercube-Based Encoding for Evolving Large-Scale Neural Networks

Neuroevolution: from architectures to learning

Three new graphical models for statistical language modelling

Modeling systems with internal state using evolino

Histograms of oriented gradients for human detection

A neural probabilistic language model

A Neural Probabilistic Language Model

Object recognition from local scale-invariant features

Long Short-Term Memory

The Inference of Regular LISP Programs from Examples

A Methodology for LISP Program Construction from Examples

Learning how to learn

Language modeling with sum-product networks

FUNDAMENTAL TECHNOLOGIES IN MODERN SPEECH RECOGNITION Digital Object Identifier 10.1109/MSP.2012.2205597

Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning

Gradient-based learning applied to document recognition

The child networks are always built and trained for 50 epochs, regardless of their structures

This method is flexible, but still need to follow some patterns (predefined recurrent structure)

This paper introduces an idea of using RNN to compose neural network architectures, and the generated structures from this paper can even outperform some state-of-the-art models

The generated structures from this methods can be generalized to other tasks

Transfer learning: Applying the generated RNN cell structure from previous tasks to the character language modeling task on the same dataset → Result is better than LSTM cell

RNN makes it possible to search in different variable-length architecture spaces and different types of structures

Field of Study

Journal Information

Name