Improved Regularization of Convolutional Neural Networks with Cutout

Published in

arXiv.org(2017)

External Links:

Generate Graph

TL;DR

This paper shows that the simple regularization technique of randomly masking out square regions of input during training, which is called cutout, can be used to improve the robustness and overall performance of convolutional neural networks.

Abstract

Convolutional neural networks are capable of learning powerful representational spaces, which are necessary for tackling complex learning tasks. However, due to the model capacity required to capture such representations, they are often susceptible to overfitting and therefore require proper regularization in order to generalize well. In this paper, we show that the simple regularization technique of randomly masking out square regions of input during training, which we call cutout, can be used to improve the robustness and overall performance of convolutional neural networks. Not only is this method extremely easy to implement, but we also demonstrate that it can be used in conjunction with existing forms of data augmentation and other regularizers to further improve model performance. We evaluate this method by applying it to current state-of-the-art architectures on the CIFAR-10, CIFAR-100, and SVHN datasets, yielding new state-of-the-art results of 2.56%, 15.20%, and 1.30% test error respectively. Code is available at this https URL

Authors

Graham W. Taylor

6 papers

Terrance Devries

2 papers

References23 items

Shake-Shake regularization

Smart Augmentation Learning an Optimal Data Augmentation Strategy

Analysis on the Dropout Effect in Convolutional Neural Networks

An Analysis of Deep Neural Network Models for Practical Applications

Wide Residual Networks

Improved Regularization of Convolutional Neural Networks with Cutout

Published in

arXiv.org(2017)

External Links:

Generate Graph

TL;DR

Abstract

Authors

Graham W. Taylor

6 papers

Terrance Devries

2 papers

References23 items

Shake-Shake regularization

Smart Augmentation Learning an Optimal Data Augmentation Strategy

Analysis on the Dropout Effect in Convolutional Neural Networks

An Analysis of Deep Neural Network Models for Practical Applications

Wide Residual Networks

Context Encoders: Feature Learning by Inpainting

Identity Mappings in Deep Residual Networks

Towards dropout training for convolutional neural networks

Deep Image: Scaling up Image Recognition

Show and tell: A neural image caption generator

Efficient object localization using Convolutional Networks

Fully convolutional networks for semantic segmentation

DeepPose: Human Pose Estimation via Deep Neural Networks

ImageNet classification with deep convolutional neural networks

Improving neural networks by preventing co-adaptation of feature detectors

An Analysis of Single-Layer Networks in Unsupervised Feature Learning

Deep Learners Benefit More from Out-of-Distribution Examples

Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion

Dropout: a simple way to prevent neural networks from overfitting

Reading Digits in Natural Images with Unsupervised Feature Learning

Learning Multiple Layers of Features from Tiny Images

GradientBased Learning Applied to Document Recognition

Gradient-based learning applied to document recognition

Field of Study

Computer Science

Journal Information

Name

ArXiv

Volume

abs/2005.00687

Venue Information

Name

arXiv.org

Type

URL

https://arxiv.org

Alternate Names

ArXiv