SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation (2015-11-02T00:00:00.000000Z)

TL;DR

Quantitative assessments show that SegNet provides good performance with competitive inference time and most efficient inference memory-wise as compared to other architectures, including FCN and DeconvNet.

Abstract

We present a novel and practical deep fully convolutional neural network architecture for semantic pixel-wise segmentation termed SegNet. This core trainable segmentation engine consists of an encoder network, a corresponding decoder network followed by a pixel-wise classification layer. The architecture of the encoder network is topologically identical to the 13 convolutional layers in the VGG16 network [1] . The role of the decoder network is to map the low resolution encoder feature maps to full input resolution feature maps for pixel-wise classification. The novelty of SegNet lies is in the manner in which the decoder upsamples its lower resolution input feature map(s). Specifically, the decoder uses pooling indices computed in the max-pooling step of the corresponding encoder to perform non-linear upsampling. This eliminates the need for learning to upsample. The upsampled maps are sparse and are then convolved with trainable filters to produce dense feature maps. We compare our proposed architecture with the widely adopted FCN [2] and also with the well known DeepLab-LargeFOV [3] , DeconvNet [4] architectures. This comparison reveals the memory versus accuracy trade-off involved in achieving good segmentation performance. SegNet was primarily motivated by scene understanding applications. Hence, it is designed to be efficient both in terms of memory and computational time during inference. It is also significantly smaller in the number of trainable parameters than other competing architectures and can be trained end-to-end using stochastic gradient descent. We also performed a controlled benchmark of SegNet and other architectures on both road scenes and SUN RGB-D indoor scene segmentation tasks. These quantitative assessments show that SegNet provides good performance with competitive inference time and most efficient inference memory-wise as compared to other architectures. We also provide a Caffe implementation of SegNet and a web demo at http://mi.eng.cam.ac.uk/projects/segnet/.

References73 items

Numerical Optimization

The Cityscapes Dataset for Semantic Urban Scene Understanding

Multi-Scale Context Aggregation by Dilated Convolutions

Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding

Understanding symmetries in deep networks

Semantic Image Segmentation via Deep Parsing Network

Decoupled Deep Neural Network for Semi-supervised Semantic Segmentation

ParseNet: Looking Wider to See Better

SUN RGB-D: A RGB-D scene understanding benchmark suite

SegNet: A Deep Convolutional Encoder-Decoder Architecture for Robust Semantic Pixel-Wise Labelling

U-Net: Convolutional Networks for Biomedical Image Segmentation

Learning Deconvolution Network for Semantic Segmentation

Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation

Fully Connected Deep Structured Networks

Conditional Random Fields as Recurrent Neural Networks

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation

Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

Vision-Based Offline-Online Perception Paradigm for Autonomous Driving

Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs

Object Detectors Emerge in Deep Scene CNNs

Feedforward semantic segmentation with zoom-out features

Hypercolumns for object segmentation and fine-grained localization

Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-scale Convolutional Architecture

Fully convolutional networks for semantic segmentation

Dense 3D semantic mapping of indoor scenes from RGB-D images

Fast Semantic Segmentation of RGB-D Scenes with GPU-Accelerated Deep Neural Networks

Going deeper with convolutions

Edge Boxes: Locating Object Proposals from Edges

Learning a Deep Convolutional Network for Image Super-Resolution

Very Deep Convolutional Networks for Large-Scale Image Recognition

ImageNet Large Scale Visual Recognition Challenge

The Pascal Visual Object Classes Challenge: A Retrospective

Neural Decision Forests for Semantic Image Labelling

The Role of Context for Object Detection and Semantic Segmentation in the Wild

Unrolling Loopy Top-Down Semantic Feedback in Convolutional Deep Networks

Recurrent Convolutional Neural Networks for Scene Labeling

Caffe: Convolutional Architecture for Fast Feature Embedding

Depth Map Prediction from a Single Image using a Multi-Scale Deep Network

Microsoft COCO: Common Objects in Context

Learning Hierarchical Features for Scene Labeling

Perceptual Organization and Recognition of Indoor Scenes from RGB-D Images

Indoor Segmentation and Support Inference from RGBD Images

Local Label Descriptor for Example Based Semantic Image Labeling

Are we ready for autonomous driving? The KITTI vision benchmark suite

RGB-(D) scene labeling: Features and algorithms

Scene parsing with Multiscale Feature Learning, Purity Trees, and Optimal Covers

Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials

Structured class-labels in random forests for semantic image labelling

Semantic contours from inverse detectors

Parsing Natural Scenes and Natural Language with Recursive Neural Networks

Learning Convolutional Feature Hierarchies for Visual Recognition

Superparsing

Semantic Segmentation of Urban Scenes Using Dense Depth Maps

What, Where and How Many? Combining Object Detectors and CRFs

LabelMe: Online Image Annotation and Applications

Deconvolutional networks

Scene Text Recognition using Higher Order Language Priors

What is the best multi-stage architecture for object recognition?

Decomposing a scene into geometric and semantically consistent regions

Semantic object classes in video: A high-definition ground truth database

Segmentation and Recognition Using Structure from Motion Point Clouds

SIFT Flow: Dense Correspondence across Different Scenes

Semantic texton forests for image categorization and segmentation

LabelMe: A Database and Web-Based Tool for Image Annotation

Unsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition

Learning to detect natural image boundaries using local brightness, color, and texture cues

SceneNet: Understanding real world indoor scenes with synthetic data,

Dropout as a Bayesian Approximation : Insights and Applications

What is a good evaluation measure for semantic segmentation?

Large-Scale Machine Learning with Stochastic Gradient Descent

Combining Appearance and Structure from Motion Features for Road Scene Understanding

Deep Convolutional Networks for Scene Parsing