Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation (2018-02-07T00:00:00.000000Z)

TL;DR

This work extends DeepLabv3 by adding a simple yet effective decoder module to refine the segmentation results especially along object boundaries and applies the depthwise separable convolution to both Atrous Spatial Pyramid Pooling and decoder modules, resulting in a faster and stronger encoder-decoder network.

Abstract

Spatial pyramid pooling module or encode-decoder structure are used in deep neural networks for semantic segmentation task. The former networks are able to encode multi-scale contextual information by probing the incoming features with filters or pooling operations at multiple rates and multiple effective fields-of-view, while the latter networks can capture sharper object boundaries by gradually recovering the spatial information. In this work, we propose to combine the advantages from both methods. Specifically, our proposed model, DeepLabv3+, extends DeepLabv3 by adding a simple yet effective decoder module to refine the segmentation results especially along object boundaries. We further explore the Xception model and apply the depthwise separable convolution to both Atrous Spatial Pyramid Pooling and decoder modules, resulting in a faster and stronger encoder-decoder network. We demonstrate the effectiveness of the proposed model on PASCAL VOC 2012 and Cityscapes datasets, achieving the test set performance of 89.0\% and 82.1\% without any post-processing. Our paper is accompanied with a publicly available reference implementation of the proposed models in Tensorflow at \url{this https URL}.

Authors

Liang-Chieh Chen

14 papers

Florian Schroff

8 papers

Hartwig Adam

15 papers

TL;DR

Abstract

Authors

References88 items

ExFuse: Enhancing Feature Fusion for Semantic Segmentation

In-place Activated BatchNorm for Memory-Optimized Training of DNNs

Deep Dual Learning for Semantic Image Segmentation

Dense and Low-Rank Gaussian CRFs Using Deep Embeddings

Densely connected deconvolutional network for semantic segmentation

Stacked Deconvolutional Network for Semantic Segmentation

Learning Transferable Architectures for Scalable Image Recognition

Scene Parsing through ADE20K Dataset

Revisiting Unreasonable Effectiveness of Data in Deep Learning Era

ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices

Learning Object Interactions and Descriptions for Semantic Image Segmentation

Gated Feedback Refinement Network for Dense Image Labeling

Rethinking Atrous Convolution for Semantic Image Segmentation

MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

Not All Pixels Are Equal: Difficulty-Aware Semantic Segmentation via Deep Layer Cascade

Deformable Convolutional Networks

Large Kernel Matters — Improve Semantic Segmentation by Global Convolutional Network

Understanding Convolution for Semantic Segmentation

DSSD : Deconvolutional Single Shot Detector

Beyond Skip Connections: Top-Down Modulation for Object Detection

COCO-Stuff: Thing and Stuff Classes in Context

Feature Pyramid Networks for Object Detection

Pyramid Scene Parsing Network

Wider or Deeper: Revisiting the ResNet Model for Visual Recognition

Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes

RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation

Aggregated Residual Transformations for Deep Neural Networks

Xception: Deep Learning with Depthwise Separable Convolutions

Design of Efficient Convolutional Layers using Single Intra-channel Convolution, Topological Subdivisioning and Spatial "Bottleneck" Structure

Gaussian Conditional Random Field Network for Semantic Segmentation

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

The Cityscapes Dataset for Semantic Urban Scene Understanding

Fast, Exact and Multi-scale Inference for Semantic Image Segmentation with Deep Gaussian CRFs

Stacked Hourglass Networks for Human Pose Estimation

TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems

Deep Residual Learning for Image Recognition

Attention to Scale: Scale-Aware Semantic Image Segmentation

SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation

Semantic Image Segmentation via Deep Parsing Network

ParseNet: Looking Wider to See Better

Modeling local and global deformations in Deep Learning: Epitomic convolution, Multiple Instance Learning, and sliding window detection

U-Net: Convolutional Networks for Biomedical Image Segmentation

Learning Deconvolution Network for Semantic Segmentation

Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation

Learning Sparse High Dimensional Filters: Image Filtering, Dense CRFs and Bilateral Neural Networks

Distilling the Knowledge in a Neural Network

Fully Connected Deep Structured Networks

Conditional Random Fields as Recurrent Neural Networks

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Weakly-and Semi-Supervised Learning of a Deep Convolutional Network for Semantic Image Segmentation

Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation

Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs

Flattened Convolutional Neural Networks for Feedforward Acceleration

Convolutional feature masking for joint object and stuff segmentation

Material recognition in the wild with the Materials in Context Database

Feedforward semantic segmentation with zoom-out features

Hypercolumns for object segmentation and fine-grained localization

Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-scale Convolutional Architecture

Fully convolutional networks for semantic segmentation

Going deeper with convolutions

Very Deep Convolutional Networks for Large-Scale Image Recognition

ImageNet Large Scale Visual Recognition Challenge

The Pascal Visual Object Classes Challenge: A Retrospective

The Role of Context for Object Detection and Semantic Segmentation in the Wild

Recurrent Convolutional Neural Networks for Scene Labeling

Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition

Microsoft COCO: Common Objects in Context

OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks

Learning Hierarchical Features for Scene Labeling

Fast image scanning with deep max-pooling convolutional neural networks

ImageNet classification with deep convolutional neural networks

Describing the scene as a whole: Joint object detection, scene classification and semantic segmentation

Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials

Semantic contours from inverse detectors

Fast High‐Dimensional Filtering Using the Permutohedral Lattice

Decomposing a scene into geometric and semantically consistent regions