Population Based Training of Neural Networks (2017-11-27T00:00:00.000000Z)

TL;DR

Population Based Training is presented, a simple asynchronous optimisation algorithm which effectively utilises a fixed computational budget to jointly optimise a population of models and their hyperparameters to maximise performance.

Abstract

Neural networks dominate the modern machine learning landscape, but their training and success still suffer from sensitivity to empirical choices of hyperparameters such as model architecture, loss function, and optimisation algorithm. In this work we present \emph{Population Based Training (PBT)}, a simple asynchronous optimisation algorithm which effectively utilises a fixed computational budget to jointly optimise a population of models and their hyperparameters to maximise performance. Importantly, PBT discovers a schedule of hyperparameter settings rather than following the generally sub-optimal strategy of trying to find a single fixed set to use for the whole course of training. With just a small modification to a typical distributed hyperparameter training framework, our method allows robust and reliable training of models. We demonstrate the effectiveness of PBT on deep reinforcement learning problems, showing faster wall-clock convergence and higher final performance of agents by optimising over a suite of hyperparameters. In addition, we show the same method can be applied to supervised learning for machine translation, where PBT is used to maximise the BLEU score directly, and also to training of Generative Adversarial Networks to maximise the Inception score of generated images. In all cases PBT results in the automatic discovery of hyperparameter schedules and model selection which results in stable training and better final performance.

Authors

Jeff Donahue

8 papers

O. Vinyals

41 papers

K. Simonyan

25 papers

TL;DR

Abstract

Authors

References56 items

HyperDrive: exploring hyperparameters with POP scheduling

Hierarchical Representations for Efficient Architecture Search

StarCraft II: A New Challenge for Reinforcement Learning

Google Vizier: A Service for Black-Box Optimization

Variational Approaches for Auto-Encoding Generative Adversarial Networks

Attention is All you Need

Improved Training of Wasserstein GANs

FeUdal Networks for Hierarchical Reinforcement Learning

Large-Scale Evolution of Image Classifiers

Bayesian Optimization with Robust Bayesian Neural Networks

Reinforcement Learning with Unsupervised Auxiliary Tasks

LR-GAN: Layered Recursive Generative Adversarial Networks for Image Generation

SGDR: Stochastic Gradient Descent with Warm Restarts

A Survey on Evolutionary Computation Approaches to Feature Selection

The Parallel Knowledge Gradient Method for Batch Bayesian Optimization

Improved Techniques for Training GANs

Convolution by Evolution: Differentiable Pattern Producing Networks

Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets

Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization

Asynchronous Methods for Deep Reinforcement Learning

Selecting Near-Optimal Learners via Incremental Data Allocation

Parallel Predictive Entropy Search for Batch Global Optimization of Expensive Objective Functions

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

Optimizing deep learning hyper-parameters through an evolutionary algorithm

Speeding Up Automatic Hyperparameter Optimization of Deep Neural Networks by Extrapolation of Learning Curves

Cyclical Learning Rates for Training Neural Networks

Batch Bayesian Optimization via Local Penalization

Scalable Bayesian Optimization Using Deep Neural Networks

Adam: A Method for Stochastic Optimization

Freeze-Thaw Bayesian Optimization

Multi-Task Bayesian Optimization

The Arcade Learning Environment: An Evaluation Platform for General Agents

Practical Bayesian Optimization of Machine Learning Algorithms

Random Search for Hyper-Parameter Optimization

Algorithms for Hyper-Parameter Optimization

Evolutionary Computation Meets Machine Learning: A Survey

Sequential Model-Based Optimization for General Algorithm Configuration

Efficient Multi-Start Strategies for Local Search Algorithms

Natural Selection Fails to Optimize Mutation Rates for Long-Term Adaptation on Rugged Fitness Landscapes

Lamarckian Evolution and the Baldwin Effect in Evolutionary Neural Networks

Optimization for problem classes-neural networks that learn to learn

G-Prop-III: Global Optimization of Multilayer Perceptrons using an Evolutionary Algorithm

An Overview of Parameter Control Methods by Self-Adaptation in Evolutionary Algorithms

Probabilistic Incremental Program Evolution: Stochastic Search Through Program Space

Exploring the effects of Lamarckian and Baldwinian learning in evolving recurrent neural networks

Empirical Investigation of the Benefits of Partial Lamarckianism

GENERATIVE ADVERSARIAL NETS

Speed learning on the ﬂy

Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural networks for machine learning

Gaussian process optimization in the bandit setting: No regret and experimental design

Self-adaptive Evolutionary Algorithms

Adapting Crossover in Evolutionary Algorithms

THE GENERALIZATION OF ‘STUDENT'S’ PROBLEM WHEN SEVERAL DIFFERENT POPULATION VARLANCES ARE INVOLVED

Trends and Applications in Constructive Approximation Evolutionary Optimization of Neural Systems: the Use of Strategy Adaptation

24th Annual Conference on Learning Theory Oracle inequalities for computationally budgeted model selection

CIFAR GAN samples generated by the best performing generator of the Baseline GAN (top) and PBT-GAN (bottom)

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names