End-to-End One-Shot Human Parsing (2021-05-04T00:00:00.000000Z)

TL;DR

An end-to-end human parsing framework that can quickly adapt to the novel classes and mitigate the testing bias issue is proposed, and a contrastive loss at the prototype level is added to enforce inter-class distances.

Abstract

Previous human parsing methods are limited to parsing humans into pre-defined classes, which is inflexible for practical fashion applications that often have new fashion item classes. In this paper, we define a novel one-shot human parsing (OSHP) task that requires parsing humans into an open set of classes defined by any test example. During training, only base classes are exposed, which only overlap with part of the test-time classes. To address three main challenges in OSHP, i.e., small sizes, testing bias, and similar parts, we devise an End-to-end One-shot human Parsing Network (EOP-Net). Firstly, an end-to-end human parsing framework is proposed to parse the query image into both coarse-grained and fine-grained human classes, which embeds rich semantic information that is shared across different granularities to identify the small-sized human classes. Then, we gradually smooth the training-time static prototypes to get robust class representations. Moreover, we employ a dynamic objective to encourage the network enhancing features’ representational capability in the early training phase while improving features’ transferability in the late training phase. Therefore, our method can quickly adapt to the novel classes and mitigate the testing bias issue. In addition, we add a contrastive loss at the prototype level to enforce inter-class distances, thereby discriminating the similar parts. For comprehensive evaluations on the new task, we tailor three existing popular human parsing benchmarks to the OSHP task. Experiments demonstrate that EOP-Net outperforms representative one-shot segmentation models by large margins and serves as a strong baseline for further research.

Authors

D. Tao

37 papers

Jing Zhang

12 papers

Jianfei Cai

8 papers

TL;DR

Abstract

Authors

References97 items

Personalize Segment Anything Model with One Shot

Segment Anything

Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformer

Learning Meta-class Memory for Few-Shot Semantic Segmentation

ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Bias

Multi-human Parsing with a Graph-based Generative Adversarial Model

Mining Latent Classes for Few-shot Segmentation

Scalable Vision Transformers with Hierarchical Pooling

Scalable Visual Transformers with Hierarchical Pooling

Training data-efficient image transformers & distillation through attention

Progressive One-shot Human Parsing

Generalized Few-shot Semantic Segmentation

Renovating Parsing R-CNN for Accurate Multiple Human Parsing

Prior Guided Feature Enrichment Network for Few-Shot Segmentation

Part-aware Prototype Network for Few-shot Semantic Segmentation

Self-paced Contrastive Learning with Hybrid Memory for Domain Adaptive Object Re-ID

Part-Aware Context Network for Human Parsing

Correlating Edge, Pose With Parsing

SimPropNet: Improved Similarity Propagation for Few-shot Image Segmentation

Self-Learning With Rectification Strategy for Human Parsing

CRNet: Cross-Reference Networks for Few-Shot Segmentation

Hierarchical Human Parsing With Typed Part-Relation Reasoning

Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learning

A New Meta-Baseline for Few-Shot Learning

A Simple Framework for Contrastive Learning of Visual Representations

Towards High Performance Human Keypoint Detection

Learning Semantic Neural Tree for Human Parsing

Grapy-ML: Graph Pyramid Mutual Learning for Cross-dataset Human Parsing

Differentiable Meta-learning Model for Few-shot Semantic Segmentation

Momentum Contrast for Unsupervised Visual Representation Learning

Self-Correction for Human Parsing

ClothFlow: A Flow-Based Model for Clothed Person Generation

Learning Compositional Neural Information Fusion for Human Parsing

Pyramid Graph Networks With Connection Attentions for Region-Based One-Shot Semantic Segmentation

FW-GAN: Flow-Navigated Warping GAN for Video Virtual Try-On

PANet: Few-Shot Image Semantic Segmentation With Prototype Alignment

Relational Generalized Few-Shot Learning

Attention-Based Multi-Context Guiding for Few-Shot Semantic Segmentation

Learning Adaptive Classifiers Synthesis for Generalized Few-Shot Learning

Fine-Grained Multi-human Parsing

Graphonomy: Universal Human Parsing via Graph Transfer Learning

CANet: Class-Agnostic Segmentation Networks With Iterative Refinement and Attentive Few-Shot Learning

AMP: Adaptive Masked Proxies for Few-Shot Segmentation

Parsing R-CNN for Instance-Level Human Analysis

M2E-Try On Net: Fashion from Model to Everyone

SG-One: Similarity Guidance Network for One-Shot Semantic Segmentation

Incremental Few-Shot Learning with Attention Attractor Networks

Multi-Human Parsing Machines

Devil in the Details: Towards Accurate Single and Multiple Human Parsing

Mutual Learning to Adapt for Joint Human Parsing and Pose Estimation

Instance-level Human Parsing via Part Grouping Network

Macro-Micro Adversarial Network for Human Parsing

Human Pose Estimation with Parsing Induced Learner

Weakly and Semi Supervised Human Body Part Parsing via Pose-Guided Knowledge Transfer

Unsupervised Feature Learning via Non-parametric Instance Discrimination

Progressive Cognitive Human Parsing

Dynamic Few-Shot Visual Learning Without Forgetting

Understanding Humans in Crowded Scenes: Deep Nested Adversarial Learning and A New Benchmark for Multi-Human Parsing

Look into Person: Joint Body Parsing & Pose Estimation Network and a New Benchmark

Conditional Networks for Few-Shot Semantic Segmentation

Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation

Low-Shot Learning with Imprinted Weights

Learning to Compare: Relation Network for Few-Shot Learning

Clothing retrieval with visual attention model

One-Shot Learning for Semantic Segmentation

Joint Multi-person Pose Estimation and Semantic Part Segmentation

Self-Supervised Neural Aggregation Networks for Human Parsing

Multiple-Human Parsing in the Wild

No Fuss Distance Metric Learning Using Proxies

Look into Person: Self-Supervised Structure-Sensitive Learning and a New Benchmark for Human Parsing

Prototypical Networks for Few-shot Learning

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

One-Shot Video Object Segmentation

Xception: Deep Learning with Depthwise Separable Convolutions

Matching Networks for One Shot Learning

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs