Panoptic Feature Pyramid Networks

Published in

Computer Vision and Pattern Recognition(2019)

External Links:

Generate Graph DownloadPDF

TL;DR

This work endsow Mask R-CNN, a popular instance segmentation method, with a semantic segmentation branch using a shared Feature Pyramid Network (FPN) backbone, and shows it is a robust and accurate baseline for both tasks.

Abstract

The recently introduced panoptic segmentation task has renewed our community's interest in unifying the tasks of instance segmentation (for thing classes) and semantic segmentation (for stuff classes). However, current state-of-the-art methods for this joint task use separate and dissimilar networks for instance and semantic segmentation, without performing any shared computation. In this work, we aim to unify these methods at the architectural level, designing a single network for both tasks. Our approach is to endow Mask R-CNN, a popular instance segmentation method, with a semantic segmentation branch using a shared Feature Pyramid Network (FPN) backbone. Surprisingly, this simple baseline not only remains effective for instance segmentation, but also yields a lightweight, top-performing method for semantic segmentation. In this work, we perform a detailed study of this minimally extended version of Mask R-CNN with FPN, which we refer to as Panoptic FPN, and show it is a robust and accurate baseline for both tasks. Given its effectiveness and conceptual simplicity, we hope our method can serve as a strong baseline and aid future research in panoptic segmentation.

Authors

Kaiming He

22 papers

Ross B. Girshick

23 papers

Piotr Dollár

12 papers

References62 items

Learning to Fuse Things and Stuff

Triply Supervised Decoder Networks for Joint Detection and Segmentation

PSANet: Point-wise Spatial Attention Network for Scene Parsing

Panoptic Segmentation with a Joint Semantic and Instance Segmentation Network

Weakly- and Semi-Supervised Panoptic Segmentation

Panoptic Feature Pyramid Networks

Published in

Computer Vision and Pattern Recognition(2019)

External Links:

Generate Graph DownloadPDF

TL;DR

Abstract

Authors

Kaiming He

22 papers

Ross B. Girshick

23 papers

Piotr Dollár

12 papers

References62 items

Learning to Fuse Things and Stuff

Triply Supervised Decoder Networks for Joint Detection and Segmentation

PSANet: Point-wise Spatial Attention Network for Scene Parsing

Panoptic Segmentation with a Joint Semantic and Instance Segmentation Network

Weakly- and Semi-Supervised Panoptic Segmentation

Alexander Kirillov

6 papers

Group Normalization

Path Aggregation Network for Instance Segmentation

Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation

Panoptic Segmentation

MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features

In-place Activated BatchNorm for Memory-Optimized Training of DNNs

Cascade R-CNN: Delving Into High Quality Object Detection

Non-local Neural Networks

MegDet: A Large Mini-Batch Object Detector

The Mapillary Vistas Dataset for Semantic Understanding of Street Scenes

SGN: Sequential Grouping Networks for Instance Segmentation

Squeeze-and-Excitation Networks

BlitzNet: A Real-Time Deep Network for Scene Understanding

Scene Parsing through ADE20K Dataset

Rethinking Atrous Convolution for Semantic Image Segmentation

BiSeg: Simultaneous Instance Segmentation and Semantic Segmentation with Fully Convolutional Networks

Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics

Pixelwise Instance Segmentation with a Dynamically Instantiated Network

Deformable Convolutional Networks

COCO-Stuff: Thing and Stuff Classes in Context

Feature Pyramid Networks for Object Detection

Pyramid Scene Parsing Network

InstanceCut: From Edges to Instances with MultiCut

Fully Convolutional Instance-Aware Semantic Segmentation

Aggregated Residual Transformations for Deep Neural Networks

UberNet: Training a Universal Convolutional Neural Network for Low-, Mid-, and High-Level Vision Using Diverse Datasets and Limited Memory

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation

Cross-Stitch Networks for Multi-task Learning

The Cityscapes Dataset for Semantic Urban Scene Understanding

Learning to Refine Object Segments

Stacked Hourglass Networks for Human Pose Estimation

Inside-Outside Net: Detecting Objects in Context with Skip Pooling and Recurrent Neural Networks

Deep Residual Learning for Image Recognition

SSD: Single Shot MultiBox Detector

Recombinator Networks: Learning Coarse-to-Fine Feature Aggregation

Multi-Scale Context Aggregation by Dilated Convolutions

SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

U-Net: Convolutional Networks for Biomedical Image Segmentation

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Fully convolutional networks for semantic segmentation

ImageNet Large Scale Visual Recognition Challenge

Scene Parsing with Object Instances and Occlusion Ordering

Microsoft COCO: Common Objects in Context

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

Describing the scene as a whole: Joint object detection, scene classification and semantic segmentation

Image Parsing: Unifying Segmentation, Detection, and Recognition