Striving for Simplicity: The All Convolutional Net (2014-12-21T00:00:00.000000Z)

TL;DR

It is found that max-pooling can simply be replaced by a convolutional layer with increased stride without loss in accuracy on several image recognition benchmarks.

Abstract

Most modern convolutional neural networks (CNNs) used for object recognition are built using the same principles: Alternating convolution and max-pooling layers followed by a small number of fully connected layers. We re-evaluate the state of the art for object recognition from small images with convolutional networks, questioning the necessity of different components in the pipeline. We find that max-pooling can simply be replaced by a convolutional layer with increased stride without loss in accuracy on several image recognition benchmarks. Following this finding -- and building on other recent work for finding simple network structures -- we propose a new architecture that consists solely of convolutional layers and yields competitive or state of the art performance on several object recognition datasets (CIFAR-10, CIFAR-100, ImageNet). To analyze the network we introduce a new variant of the "deconvolution approach" for visualizing features learned by CNNs, which can be applied to a broader range of network structures than existing approaches.

Authors

Martin A. Riedmiller

7 papers

T. Brox

22 papers

Alexey Dosovitskiy

16 papers

TL;DR

Abstract

Authors

References29 items

On orthogonality and learning recurrent networks with long term dependencies

Fractional Max-Pooling

Deeply-Supervised Nets

Going deeper with convolutions

Very Deep Convolutional Networks for Large-Scale Image Recognition

Deep Networks with Internal Selective Attention through Feedback Connections

Caffe: Convolutional Architecture for Fast Feature Embedding

Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps

Improving Deep Neural Networks with Probabilistic Maxout Units

Network In Network

Discriminative Transfer Learning with Tree-based Priors

Compete to Compute

Signal recovery from Pooling Representations

Visualizing and Understanding Convolutional Networks

Learned-Norm Pooling for Deep Feedforward and Recurrent Neural Networks

Regularization of Neural Networks using DropConnect

Maxout Networks

Stochastic Pooling for Regularization of Deep Convolutional Neural Networks

ImageNet classification with deep convolutional neural networks

Improving neural networks by preventing co-adaptation of feature detectors

Beyond spatial pyramids: Receptive field learning for pooled image features

High-Performance Neural Networks for Visual Object Classification

What is the best multi-stage architecture for object recognition?

ImageNet: A large-scale hierarchical image database

Hierarchical Neural Networks for Image Interpretation

Accepted as a workshop contribution

Dropout: a simple way to prevent neural networks from overfitting

Learning Multiple Layers of Features from Tiny Images

Gradient-based learning applied to document recognition

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names