Network In Network (2013-12-16T00:00:00.000000Z)

TL;DR

With enhanced local modeling via the micro network, the proposed deep network structure NIN is able to utilize global average pooling over feature maps in the classification layer, which is easier to interpret and less prone to overfitting than traditional fully connected layers.

Abstract

We propose a novel deep network structure called "Network In Network" (NIN) to enhance model discriminability for local patches within the receptive field. The conventional convolutional layer uses linear filters followed by a nonlinear activation function to scan the input. Instead, we build micro neural networks with more complex structures to abstract the data within the receptive field. We instantiate the micro neural network with a multilayer perceptron, which is a potent function approximator. The feature maps are obtained by sliding the micro networks over the input in a similar manner as CNN; they are then fed into the next layer. Deep NIN can be implemented by stacking mutiple of the above described structure. With enhanced local modeling via the micro network, we are able to utilize global average pooling over feature maps in the classification layer, which is easier to interpret and less prone to overfitting than traditional fully connected layers. We demonstrated the state-of-the-art classification performances with NIN on CIFAR-10 and CIFAR-100, and reasonable performances on SVHN and MNIST datasets.

Authors

Shuicheng Yan

14 papers

Min Lin

1 papers

Qiang Chen

1 papers

Network In Network

TL;DR

Abstract

Authors

References20 items

Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks

Discriminative Transfer Learning with Tree-based Priors

Visualizing and Understanding Convolutional Networks

Learning Hierarchical Features for Scene Labeling

Regularization of Neural Networks using DropConnect

Piecewise Linear Multilayer Perceptrons and Dropout

Knowledge Matters: Importance of Prior Information for Optimization

Stochastic Pooling for Regularization of Deep Convolutional Neural Networks

Learnable Pooling Regions for Image Classification

ImageNet classification with deep convolutional neural networks

Improving neural networks by preventing co-adaptation of feature detectors

Representation Learning: A Review and New Perspectives

Practical Bayesian Optimization of Machine Learning Algorithms

ICA with Reconstruction Cost for Efficient Overcomplete Feature Learning

Human Face Detection in Visual Scenes

PRINCIPLES OF NEURODYNAMICS. PERCEPTRONS AND THE THEORY OF BRAIN MECHANISMS

Improving Neural Networks with Dropout

Reading Digits in Natural Images with Unsupervised Feature Learning

Learning Multiple Layers of Features from Tiny Images

Gradient-based learning applied to document recognition

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names