AutoAugment: Learning Augmentation Policies from Data (2018-05-24T00:00:00.000000Z)

TL;DR

This paper describes a simple procedure called AutoAugment to automatically search for improved data augmentation policies, which achieves state-of-the-art accuracy on CIFAR-10, CIFar-100, SVHN, and ImageNet (without additional data).

Abstract

Data augmentation is an effective technique for improving the accuracy of modern image classifiers. However, current data augmentation implementations are manually designed. In this paper, we describe a simple procedure called AutoAugment to automatically search for improved data augmentation policies. In our implementation, we have designed a search space where a policy consists of many sub-policies, one of which is randomly chosen for each image in each mini-batch. A sub-policy consists of two operations, each operation being an image processing function such as translation, rotation, or shearing, and the probabilities and magnitudes with which the functions are applied. We use a search algorithm to find the best policy such that the neural network yields the highest validation accuracy on a target dataset. Our method achieves state-of-the-art accuracy on CIFAR-10, CIFAR-100, SVHN, and ImageNet (without additional data). On ImageNet, we attain a Top-1 accuracy of 83.5% which is 0.4% better than the previous record of 83.1%. On CIFAR-10, we achieve an error rate of 1.5%, which is 0.6% better than the previous state-of-the-art. Augmentation policies we find are transferable between datasets. The policy learned on ImageNet transfers well to achieve significant improvements on other datasets, such as Oxford Flowers, Caltech-101, Oxford-IIT Pets, FGVC Aircraft, and Stanford Cars.

Authors

Quoc V. Le

42 papers

E. D. Cubuk

9 papers

Dandelion Mané

3 papers

TL;DR

Abstract

Authors

References78 items

Do CIFAR-10 Classifiers Generalize to CIFAR-10?

Parallel Architecture and Hyperparameter Search via Successive Halving and Classification

Do Better ImageNet Models Transfer Better?

Exploring the Limits of Weakly Supervised Pretraining

Simple random search provides a competitive approach to reinforcement learning

ShakeDrop regularization

Efficient Neural Architecture Search via Parameter Sharing

Regularized Evolution for Image Classifier Architecture Search

Data Augmentation by Pairing Samples for Images Classification

Progressive Neural Architecture Search

The Effectiveness of Data Augmentation in Image Classification using Deep Learning

Simple And Efficient Architecture Search for Convolutional Neural Networks

Data Augmentation Generative Adversarial Networks

Intriguing Properties of Adversarial Examples

Data Augmentation in Emotion Classification Using Generative Adversarial Networks

Hierarchical Representations for Efficient Architecture Search

A Bayesian Data Augmentation Approach for Learning Deep Models

mixup: Beyond Empirical Risk Minimization

Learning to Compose Domain-Specific Transformations for Data Augmentation

Squeeze-and-Excitation Networks

SMASH: One-Shot Model Architecture Search through HyperNetworks

Random Erasing Data Augmentation

Improved Regularization of Convolutional Neural Networks with Cutout

Neural Optimizer Search with Reinforcement Learning

Learning Transferable Architectures for Scalable Image Recognition

Deep Layer Aggregation

Proximal Policy Optimization Algorithms

Representations in neural network based empirical potentials.

SchNet: A continuous-filter convolutional neural network for modeling quantum interactions

Shake-Shake regularization

Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning

Neural Message Passing for Quantum Chemistry

Smart Augmentation Learning an Optimal Data Augmentation Strategy

Genetic CNN

Large-Scale Evolution of Image Classifiers

Incorporating intra-class variance to fine-grained visual recognition

Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results

Dataset Augmentation in Feature Space

Aggregated Residual Transformations for Deep Neural Networks

RenderGAN: Generating Realistic Labeled Data

Designing Neural Network Architectures using Reinforcement Learning

Neural Architecture Search with Reinforcement Learning

Deep Pyramidal Residual Networks

Temporal Ensembling for Semi-Supervised Learning

SGDR: Stochastic Gradient Descent with Warm Restarts

Relationship between local structure and relaxation in out-of-equilibrium glassy systems

Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning

Wide Residual Networks

Structural Properties of Defects in Glassy Liquids.

Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning

Deep Residual Learning for Image Recognition

APAC: Augmented PAttern Classification with Neural Networks

Going deeper with convolutions

Very Deep Convolutional Networks for Large-Scale Image Recognition

Generative Adversarial Nets

Some Improvements on Deep Convolutional Neural Network Based Image Classification

Fine-Grained Visual Classification of Aircraft

Regularization of Neural Networks using DropConnect

ImageNet classification with deep convolutional neural networks

Multi-column deep neural networks for image classification

What is the best multi-stage architecture for object recognition?

ImageNet: A large-scale hierarchical image database

Automated Flower Classification over a Large Number of Classes

Generalized neural-network representation of high-dimensional potential-energy surfaces.

Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories

Best practices for convolutional neural networks applied to visual document analysis

Long Short-Term Memory

Realistic Evaluation of Semi-Supervised Learning Algorithms

Requests For Research 2

Generative Adversarial Network Based Acoustic Scene Training Set Augmentation and Selection Using SVM Hyper-Plane

Collecting a Large-scale Dataset of Fine-grained Cars

Reading Digits in Natural Images with Unsupervised Feature Learning

Learning Multiple Layers of Features from Tiny Images

Gradient-based learning applied to document recognition

Document image defect models

Neocognitron: A Self-Organizing Neural Network Model for a Mechanism of Visual Pattern Recognition