Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition (2014-06-18T00:00:00.000000Z)

TL;DR

This work equips the networks with another pooling strategy, “spatial pyramid pooling”, to eliminate the above requirement, and develops a new network structure, called SPP-net, which can generate a fixed-length representation regardless of image size/scale.

Abstract

Existing deep convolutional neural networks (CNNs) require a fixed-size (e.g., 224<inline-formula><tex-math>$\times$ </tex-math><alternatives><inline-graphic xlink:type="simple" xlink:href="he-ieq1-2389824.gif"/></alternatives></inline-formula>224) input image. This requirement is “artificial” and may reduce the recognition accuracy for the images or sub-images of an arbitrary size/scale. In this work, we equip the networks with another pooling strategy, “spatial pyramid pooling”, to eliminate the above requirement. The new network structure, called SPP-net, can generate a fixed-length representation regardless of image size/scale. Pyramid pooling is also robust to object deformations. With these advantages, SPP-net should in general improve all CNN-based image classification methods. On the ImageNet 2012 dataset, we demonstrate that SPP-net boosts the accuracy of a variety of CNN architectures despite their different designs. On the Pascal VOC 2007 and Caltech101 datasets, SPP-net achieves state-of-the-art classification results using a single full-image representation and no fine-tuning. The power of SPP-net is also significant in object detection. Using SPP-net, we compute the feature maps from the entire image only once, and then pool features in arbitrary regions (sub-images) to generate fixed-length representations for training the detectors. This method avoids repeatedly computing the convolutional features. In processing test images, our method is 24-102 <inline-formula><tex-math>$\times$</tex-math><alternatives><inline-graphic xlink:type="simple" xlink:href="he-ieq2-2389824.gif"/> </alternatives></inline-formula> faster than the R-CNN method, while achieving better or comparable accuracy on Pascal VOC 2007. In ImageNet Large Scale Visual Recognition Challenge (ILSVRC) 2014, our methods rank #2 in object detection and #3 in image classification among all 38 teams. This manuscript also introduces the improvement made for this competition.

References46 items

Going deeper with convolutions

Edge Boxes: Locating Object Proposals from Edges

Very Deep Convolutional Networks for Large-Scale Image Recognition

ImageNet Large Scale Visual Recognition Challenge

Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks

DeepFace: Closing the Gap to Human-Level Performance in Face Verification

Return of the Devil in the Details: Delving Deep into Convolutional Nets

Generic Object Detection with Dense Neural Patterns and Regionlets

Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation

CNN Features Off-the-Shelf: An Astounding Baseline for Recognition

Multi-scale Orderless Pooling of Deep Convolutional Activation Features

OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks

Fast Training of Convolutional Networks through FFTs

Some Improvements on Deep Convolutional Neural Network Based Image Classification

Network In Network

Deep Neural Networks for Object Detection

Regionlets for Generic Object Detection

PANDA: Pose Aligned Networks for Deep Attribute Modeling

Visualizing and Understanding Convolutional Networks

Visualizing and Understanding Convolutional Neural Networks

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition

ImageNet classification with deep convolutional neural networks

Aggregating Local Image Descriptors into Compact Codes

Segmentation as selective search for object recognition

The Importance of Encoding Versus Training with Sparse Coding and Vector Quantization

LIBSVM: A library for support vector machines

Improving the Fisher Kernel for Large-Scale Image Classification

Object Detection with Discriminatively Trained Part Based Models

Locality-constrained Linear Coding for image classification

Linear spatial pyramid matching using sparse coding for image classification

ImageNet: A large-scale hierarchical image database

Kernel Codebooks for Scene Categorization

Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories

The pyramid match kernel: discriminative classification with sets of image features

Histograms of oriented gradients for human detection

Distinctive Image Features from Scale-Invariant Keypoints

Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories

Video Google: a text retrieval approach to object matching in videos

Backpropagation Applied to Handwritten Zip Code Recognition

Caffe: An open source convolutional architecture for fast feature embedding

The devil is in the details: an evaluation of recent feature encoding methods

Comput. Vis. Pattern Recognit

The PASCAL Visual Object Classes Challenge

The Pascal Visual Object Classes Challenge 2006 ( VOC 2006 ) Results

International Journal of Computer Vision manuscript No. (will be inserted by the editor) The PASCAL Visual Object Classes (VOC) Challenge