Part-aware Prototype Network for Few-shot Semantic Segmentation (2020-07-13T00:00:00.000000Z)

TL;DR

A novel few-shot semantic segmentation framework based on the prototype representation, capable of capturing diverse and fine-grained object features, and a novel graph neural network model to generate and enhance the proposed part-aware prototypes based on labeled and unlabeled images.

Abstract

Few-shot semantic segmentation aims to learn to segment new object classes with only a few annotated examples, which has a wide range of real-world applications. Most existing methods either focus on the restrictive setting of one-way few-shot segmentation or suffer from incomplete coverage of object regions. In this paper, we propose a novel few-shot semantic segmentation framework based on the prototype representation. Our key idea is to decompose the holistic class representation into a set of part-aware prototypes, capable of capturing diverse and fine-grained object features. In addition, we propose to leverage unlabeled data to enrich our part-aware prototypes, resulting in better modeling of intra-class variations of semantic objects. We develop a novel graph neural network model to generate and enhance the proposed part-aware prototypes based on labeled and unlabeled images. Extensive experimental evaluations on two benchmarks show that our method outperforms the prior art with a sizable margin.

Authors

Xuming He

6 papers

Songyang Zhang

4 papers

Yongfei Liu

3 papers

TL;DR

Abstract

Authors

References40 items

Differentiable Meta-learning Model for Few-shot Semantic Segmentation

Pyramid Graph Networks With Connection Attentions for Region-Based One-Shot Semantic Segmentation

Feature Weighting and Boosting for Few-Shot Segmentation

PANet: Few-Shot Image Semantic Segmentation With Prototype Alignment

A Dual Attention Network with Semantic Embedding for Few-Shot Learning

Learning to Self-Train for Semi-Supervised Few-Shot Classification

CANet: Class-Agnostic Segmentation Networks With Iterative Refinement and Attentive Few-Shot Learning

Semi-Supervised Few-Shot Learning with Local and Global Consistency

Adaptive Masked Weight Imprinting for Few-Shot Segmentation

AMP: Adaptive Masked Proxies for Few-Shot Segmentation

SG-One: Similarity Guidance Network for One-Shot Semantic Segmentation

Learning to Propagate Labels: Transductive Propagation Network for Few-Shot Learning

Few-Shot Segmentation Propagation with Guided Networks

Meta-Learning for Semi-Supervised Few-Shot Classification

Conditional Networks for Few-Shot Semantic Segmentation

Learning Deep Representations of Medical Images using Siamese CNNs with Application to Content-Based Image Retrieval

Few-Shot Learning with Graph Neural Networks

Graph Attention Networks

One-Shot Learning for Semantic Segmentation

Semantic Instance Segmentation for Autonomous Driving

Rethinking Atrous Convolution for Semantic Image Segmentation

Attention is All you Need

Prototypical Networks for Few-shot Learning

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

Pyramid Scene Parsing Network

Optimization as a Model for Few-Shot Learning

Semi-Supervised Classification with Graph Convolutional Networks

Matching Networks for One Shot Learning

Deep Residual Learning for Image Recognition

SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation

Fully convolutional networks for semantic segmentation

ImageNet Large Scale Visual Recognition Challenge

Microsoft COCO: Common Objects in Context

Semantic contours from inverse detectors

A new model for learning in graph domains

Few-Shot Semantic Segmentation with Prototype Learning

The Graph Neural Network Model

Both authors contributed equally to the work

Table 1. Averaged binary-IoU over 4 folds of 1-way setting on PASCAL-5 i

Fast slic

Field of Study

Venue Information

Name

Type

URL

Alternate Names