Semantic Flow for Fast and Accurate Scene Parsing (2020-02-24T00:00:00.000000Z)

TL;DR

This paper proposes a Flow Alignment Module (FAM) to learn Semantic Flow between feature maps of adjacent levels, and broadcast high-level features to high resolution features effectively and efficiently and exhibits superior performance over other real-time methods even on light-weight backbone networks.

Abstract

In this paper, we focus on designing effective method for fast and accurate scene parsing. A common practice to improve the performance is to attain high resolution feature maps with strong semantic representation. Two strategies are widely used---atrous convolutions and feature pyramid fusion, are either computation intensive or ineffective. Inspired by the Optical Flow for motion alignment between adjacent video frames, we propose a Flow Alignment Module (FAM) to learn Semantic Flow between feature maps of adjacent levels, and broadcast high-level features to high resolution features effectively and efficiently. Furthermore, integrating our module to a common feature pyramid structure exhibits superior performance over other real-time methods even on light-weight backbone networks, such as ResNet-18. Extensive experiments are conducted on several challenging datasets, including Cityscapes, PASCAL Context, ADE20K and CamVid. Especially, our network is the first to achieve 80.4\% mIoU on Cityscapes with a frame rate of 26 FPS. The code is available at \url{this https URL}.

Authors

Zhen Zhu

3 papers

Xiangtai Li

8 papers

Ansheng You

1 papers

TL;DR

Abstract

Authors

References75 items

Spatial Pyramid Based Graph Reasoning for Semantic Segmentation

Dynamic Multi-Scale Filters for Semantic Segmentation

Dual Graph Convolutional Network for Semantic Segmentation

Boundary-Aware Feature Propagation for Scene Segmentation

SPGNet: Semantic Prediction Guidance for Scene Parsing

Asymmetric Non-Local Neural Networks for Semantic Segmentation

Dynamic Graph Message Passing Networks

Expectation-Maximization Attention Networks for Semantic Segmentation

Customizable Architecture Search for Semantic Segmentation

Co-Occurrent Features in Semantic Segmentation

Adaptive Pyramid Context Network for Semantic Segmentation

GFF: Gated Fully Fusion for Semantic Segmentation

DFANet: Deep Feature Aggregation for Real-Time Semantic Segmentation

In Defense of Pre-Trained ImageNet Architectures for Real-Time Semantic Segmentation of Road-Driving Images

Partial Order Pruning: For Best Speed/Accuracy Trade-Off in Neural Architecture Search

Panoptic Feature Pyramid Networks

Improving Semantic Segmentation via Video Propagation and Label Relaxation

ESPNetv2: A Light-Weight, Power Efficient, and General Purpose Convolutional Neural Network

CCNet: Criss-Cross Attention for Semantic Segmentation

Fast Neural Architecture Search of Compact Semantic Segmentation Models via Auxiliary Cells

Dual Attention Network for Scene Segmentation

PSANet: Point-wise Spatial Attention Network for Scene Parsing

OCNet: Object Context Network for Scene Parsing

BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation

ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design

Unified Perceptual Parsing for Scene Understanding

Guided Upsampling Network for Real-Time Semantic Segmentation

DenseASPP for Semantic Segmentation in Street Scenes

Context Contrasted Feature and Gated Multi-scale Aggregation for Scene Segmentation

Learning a Discriminative Feature Network for Semantic Segmentation

Low-Latency Video Semantic Segmentation

Adaptive Affinity Fields for Semantic Segmentation

Context Encoding for Semantic Segmentation

ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation

Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation

Non-local Neural Networks

Automatic differentiation in PyTorch

The Mapillary Vistas Dataset for Semantic Understanding of Street Scenes

Scale-Adaptive Convolutions for Scene Parsing

Semantic Video CNNs Through Representation Warping

Scene Parsing through ADE20K Dataset

Semantic Segmentation via Structured Patch Prediction, Context CRF and Guidance CRF

Rethinking Atrous Convolution for Semantic Image Segmentation

Attention is All you Need

Recurrent Scene Parsing with Perspective Understanding in the Loop

ICNet for Real-Time Semantic Segmentation on High-Resolution Images

Unsupervised Learning of Depth and Ego-Motion from Video

Deformable Convolutional Networks

Semantic Video Segmentation by Gated Recurrent Flow Propagation

Feature Pyramid Networks for Object Detection

Pyramid Scene Parsing Network

Wider or Deeper: Revisiting the ResNet Model for Visual Recognition

Deep Feature Flow for Video Recognition

Semi-Supervised Classification with Graph Convolutional Networks

Semantic Understanding of Scenes Through the ADE20K Dataset

ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

Training Region-Based Object Detectors with Online Hard Example Mining

The Cityscapes Dataset for Semantic Urban Scene Understanding

Deep Residual Learning for Image Recognition

Multi-Scale Context Aggregation by Dilated Convolutions

SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation

Spatial Transformer Networks

U-Net: Convolutional Networks for Biomedical Image Segmentation

FlowNet: Learning Optical Flow with Convolutional Networks

Fully convolutional networks for semantic segmentation

ImageNet Large Scale Visual Recognition Challenge

The Role of Context for Object Detection and Semantic Segmentation in the Wild

Semantic object classes in video: A high-definition ground truth database

A Database and Evaluation Methodology for Optical Flow

High Accuracy Optical Flow Estimation Based on a Theory for Warping

ERFNet: Efficient Residual Factorized ConvNet for Real-Time Semantic Segmentation

Beyond Grids: Learning Graph Representations for Visual Recognition

Edinburgh Research Explorer The PASCAL Visual Object Classes (VOC) Challenge

International Journal of Computer Vision manuscript No. (will be inserted by the editor) The PASCAL Visual Object Classes (VOC) Challenge

Field of Study