Foreground-Aware Stylization and Consensus Pseudo-Labeling for Domain Adaptation of First-Person Hand Segmentation (2021-07-06T00:00:00.000000Z)

TL;DR

This work proposes foreground-aware image stylization and consensus pseudo-labeling for domain adaptation of hand segmentation and demonstrates promising results in challenging multi-target domain adaptation and domain generalization settings.

Abstract

Hand segmentation is a crucial task in first-person vision. Since first-person images exhibit strong bias in appearance among different environments, adapting a pre-trained segmentation model to a new domain is required in hand segmentation. Here, we focus on appearance gaps for hand regions and backgrounds separately. We propose (i) foreground-aware image stylization and (ii) consensus pseudo-labeling for domain adaptation of hand segmentation. We stylize source images independently for the foreground and background using target images as style. To resolve the domain shift that the stylization has not addressed, we apply careful pseudo-labeling by taking a consensus between the models trained on the source and stylized source images. We validated our method on domain adaptation of hand segmentation from real and simulation images. Our method achieved state-of-the-art performance in both settings. We also demonstrated promising results in challenging multi-target domain adaptation and domain generalization settings.

Authors

Y. Ushiku

7 papers

Takuma Yagi

4 papers

Yoichi Sato

4 papers

TL;DR

Abstract

Authors

References67 items

Ego2Hands: A Dataset for Egocentric Two-hand Segmentation and Detection

Generalizing Hand Segmentation in Egocentric Videos With Uncertainty-Guided Model Adaptation

Understanding Human Hands in Contact at Internet Scale

Keep it Simple: Image Statistics Matching for Domain Adaptation

Unsupervised Intra-Domain Adaptation for Semantic Segmentation Through Self-Supervision

See What I See: Enabling User-Centric Robotic Assistance Using First-Person Demonstrations

Learning Texture Invariant Representation for Domain Adaptation of Semantic Segmentation

Augmented Cyclic Consistency Regularization for Unpaired Image-to-Image Translation

Hand-Priming in Object Localization for Assistive Egocentric Vision

Domain Stylization: A Fast Covariance Matching Framework Towards Domain Adaptation

Partially-Shared Variational Auto-encoders for Unsupervised Domain Adaptation with Target Shift

Analysis of the Hands in Egocentric Vision: A Survey

PyTorch: An Imperative Style, High-Performance Deep Learning Library

Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization Without Accessing Target Domain Data

Adapting Semantic Segmentation of Urban Scenes via Mask-Aware Gated Discriminator

Blending-Target Domain Adaptation by Adversarial Meta-Adaptation Networks

Bidirectional Learning for Domain Adaptation of Semantic Segmentation

Learning Joint Reconstruction of Hands and Manipulated Objects

Strong-Weak Distribution Alignment for Adaptive Object Detection

ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation

Unsupervised Multi-Target Domain Adaptation: An Information Theoretic Approach

ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness

In the Eye of Beholder: Joint Learning of Gaze and Actions in First Person Video

Unsupervised Domain Adaptation for Semantic Segmentation via Class-Balanced Self-training

DCAN: Dual Channel-wise Alignment Networks for Unsupervised Scene Adaptation

Analysis of Hand Segmentation in the Wild

A Closed-form Solution to Photorealistic Image Stylization

PrivacEye: Privacy-Preserving First-Person Vision Using Image Features and Eye Movement Analysis

Maximum Classifier Discrepancy for Unsupervised Domain Adaptation

Learning from Synthetic Data: Addressing Domain Shift for Semantic Segmentation

CyCADA: Cycle-Consistent Adversarial Domain Adaptation

The “Something Something” Video Database for Learning and Evaluating Visual Common Sense

Conditional Adversarial Domain Adaptation

An Ego-Vision System for Hand Grasp Analysis

The 2017 DAVIS Challenge on Video Object Segmentation

Deep Photo Style Transfer

Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization

Asymmetric Tri-training for Unsupervised Domain Adaptation

Adversarial Discriminative Domain Adaptation

Demystifying Neural Style Transfer

RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation

Image Style Transfer Using Convolutional Neural Networks

A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation

Domain Adaptation with Conditional Transferable Components

Privacy-Preserving Human Activity Recognition from Extreme Low Resolution

Unsupervised Domain Adaptation with Residual Transfer Networks

Lending A Hand: Detecting Hands and Recognizing Activities in Complex Egocentric Interactions

Toward Storytelling From Visual Lifelogging: An Overview

Enabling privacy-preserving first-person cameras using low-power sensors

Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views

The Yale human grasping dataset: Grasp, object, and task data in household and machine shop environments

Adam: A Method for Stochastic Optimization

Unsupervised Domain Adaptation by Backpropagation

Microsoft COCO: Common Objects in Context

Pixel-Level Hand Detection in Ego-centric Videos

Domain Adaptation under Target and Conditional Shift

First-Person Vision

Co-Training for Domain Adaptation

Understanding egocentric activities

A theory of learning from different domains

Hand Segmentation for Augmented Reality System

Graspit! A versatile simulator for robotic grasping

"GrabCut": interactive foreground extraction using iterated graph cuts

Style adapted database: Generalizing hand segmentation via semantics-aware stylization

Unpaired image-to-image translation using cycle-consistent adversarial networks

Volume 9

and as an in

Field of Study

Journal Information

Name

Page

Volume

Venue Information

Name

Type

URL