Mask as Supervision: Leveraging Unified Mask Information for Unsupervised 3D Pose Estimation (2023-12-12T00:00:00.000000Z)

TL;DR

This work proposes a unified framework that leverages mask as supervision for unsupervised 3D pose estimation that employs skeleton and physique representations that exploit accurate pose information from coarse to fine.

Abstract

Automatic estimation of 3D human pose from monocular RGB images is a challenging and unsolved problem in computer vision. In a supervised manner, approaches heavily rely on laborious annotations and present hampered generalization ability due to the limited diversity of 3D pose datasets. To address these challenges, we propose a unified framework that leverages mask as supervision for unsupervised 3D pose estimation. With general unsupervised segmentation algorithms, the proposed model employs skeleton and physique representations that exploit accurate pose information from coarse to fine. Compared with previous unsupervised approaches, we organize the human skeleton in a fully unsupervised way which enables the processing of annotation-free data and provides ready-to-use estimation results. Comprehensive experiments demonstrate our state-of-the-art pose estimation performance on Human3.6M and MPI-INF-3DHP datasets. Further experiments on in-the-wild datasets also illustrate the capability to access more data to boost our model. Code will be available at https://github.com/Charrrrrlie/Mask-as-Supervision.

Authors

Yuchen Yang

1 papers

Yu Qiao

1 papers

Xiao Sun

1 papers

TL;DR

Abstract

Authors

References57 items

SMPL: A Skinned Multi-Person Linear Model

Bootstrapping Objectness from Videos by Relaxed Common Fate and Visual Grouping

Self-supervised 3D Human Pose Estimation from a Single Image

Segment Anything

Few-Shot Geometry-Aware Keypoint Localization

VR-HandNet: A Visually and Physically Plausible Hand Manipulation System in Virtual Reality

BKinD-3D: Self-Supervised 3D Keypoint Discovery from Multi-View Videos

Unsupervised 3D Keypoint Discovery with Multi-View Geometry

TAX-Pose: Task-Specific Cross-Pose Estimation for Robot Manipulation

Fast and robust video-based exercise classification via body pose tracking and scalable multivariate time series classifiers

ARHPE: Asymmetric Relation-Aware Representation Learning for Head Pose Estimation in Industrial Human–Computer Interaction

AutoLink: Self-supervised Learning of Human Skeletons and Object Outlines by Linking Keypoints

Guess What Moves: Unsupervised Video and Image Segmentation by Anticipating Motion

Towards Alleviating the Modeling Ambiguity of Unsupervised Monocular 3D Human Pose Estimation

Learning High Fidelity Depths of Dressed Humans by Watching Social Media Dance Videos

Unsupervised Human Pose Estimation through Transforming Shape Templates

PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation

Differentiable Drawing and Sketching

Collision-free human-robot collaboration based on context awareness

Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans

Temporal Representation Learning on Monocular Videos for 3D Human Pose Estimation

CanonPose: Self-Supervised Monocular 3D Human Pose Estimation in the Wild

Self-Supervised Learning of Interpretable Keypoints From Unlabelled Videos

ACTIVATION FUNCTIONS IN NEURAL NETWORKS

Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image Synthesis

Geometry-Driven Self-Supervised Method for 3D Human Pose Estimation

Kinematic-Structure-Preserved Representation for Unsupervised 3D Human Pose Estimation

First Order Motion Model for Image Animation

Virtual reality in manufacturing: immersive and collaborative artificial-reality in design of human-robot workspace

AI Coach: Deep Human Pose Estimation and Analysis for Personalized Athletic Training Assistance

Unsupervised Learning of Landmarks by Descriptor Vector Exchange

Human–robot interaction in industrial collaborative robotics: a literature review of the decade 2008–2017

Unsupervised 3D Pose Estimation With Geometric Self-Supervision

Unsupervised Part-Based Disentangling of Object Shape and Appearance

Neural Scene Decomposition for Multi-Person Motion Capture

Self-Supervised Learning of 3D Human Pose Using Multi-View Geometry

Gaussian Mixture Models

Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning

Unsupervised Learning of Object Landmarks through Conditional Image Generation

Unsupervised Discovery of Object Landmarks as Structural Representations

Group Normalization

Integral Human Pose Regression

Unsupervised Learning of Disentangled Representations from Video

Monocular 3D Human Pose Estimation in the Wild Using Improved CNN Supervision

Stacked Hourglass Networks for Human Pose Estimation

U-Net: Convolutional Networks for Biomedical Image Segmentation

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments

2D Human Pose Estimation: New Benchmark and State of the Art Analysis

Graph Cuts and Efficient N-D Image Segmentation

Dynamic Textures

Adaptive background mixture models for real-time tracking

New geodosic distance transforms for gray-scale images

A method for stochastic optimization

Fast Marching Methods

Multiple View Geometry in Computer Vision

Toward human-centered ai: a perspective human-computer interaction

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names