Whole-Body Human Pose Estimation in the Wild (2020-07-23T00:00:00.000000Z)

TL;DR

COCO-WholeBody is introduced which extends COCO dataset with whole-body annotations and is the first benchmark that has manual annotations on the entire human body, including 133 dense landmarks with 68 on the face, 42 on hands and 23 on the body and feet.

Abstract

This paper investigates the task of 2D human whole-body pose estimation, which aims to localize dense landmarks on the entire human body including face, hands, body, and feet. As existing datasets do not have whole-body annotations, previous methods have to assemble different deep models trained independently on different datasets of the human face, hand, and body, struggling with dataset biases and large model complexity. To fill in this blank, we introduce COCO-WholeBody which extends COCO dataset with whole-body annotations. To our best knowledge, it is the first benchmark that has manual annotations on the entire human body, including 133 dense landmarks with 68 on the face, 42 on hands and 23 on the body and feet. A single-network model, named ZoomNet, is devised to take into account the hierarchical structure of the full human body to solve the scale variation of different body parts of the same person. ZoomNet is able to significantly outperform existing methods on the proposed COCO-WholeBody dataset. Extensive experiments show that COCO-WholeBody not only can be used to train deep models from scratch for whole-body pose estimation but also can serve as a powerful pre-training dataset for many different tasks such as facial landmark detection and hand keypoint estimation. The dataset is publicly available at this https URL.

Authors

Wanli Ouyang

26 papers

P. Luo

15 papers

Can Wang

2 papers

TL;DR

Abstract

Authors

References72 items

Mask-Pose Cascaded CNN for 2D Hand Pose Estimation From Single Color Image

TRB: A Novel Triplet Representation for Understanding 2D Human Body

Single-Network Whole-Body Pose Estimation

FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape From Single RGB Images

High-Resolution Representations for Labeling Pixels and Regions

Multi-Person Articulated Tracking With Spatial and Temporal Embeddings

Deep High-Resolution Representation Learning for Human Pose Estimation

OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields

Monocular Total Capture: Posing Face, Body, and Hands in the Wild

CrowdPose: Efficient Crowded Scenes Pose Estimation and a New Benchmark

A Deeply-Initialized Coarse-to-fine Ensemble of Regression Trees for Face Alignment

Pose Partition Networks for Multi-person Pose Estimation

CornerNet: Detecting Objects as Paired Keypoints

Look at Boundary: A Boundary-Aware Face Alignment Algorithm

A Cascaded Inception of Inception Network With Attention Modulated Feature Fusion for Human Pose Estimation

Simple Baselines for Human Pose Estimation and Tracking

PersonLab: Person Pose Estimation and Instance Segmentation with a Bottom-Up, Part-Based, Geometric Embedding Model

DensePose: Dense Human Pose Estimation in the Wild

Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies

GANerated Hands for Real-Time 3D Hand Tracking from Monocular RGB

Cascaded Pyramid Network for Multi-person Pose Estimation

Embodied hands

AI Challenger : A Large-scale Dataset for Going Deeper in Image Understanding

PoseTrack: A Benchmark for Human Pose Estimation and Tracking

Benchmarking and Error Diagnosis in Multi-instance Pose Estimation

Large-scale Multiview 3D Hand Pose Dataset

Deep Alignment Network: A Convolutional Neural Network for Robust Face Alignment

Generative Partition Networks for Multi-Person Pose Estimation

Learning to Estimate 3D Hand Pose from Single RGB Images

Hand Keypoint Detection in Single Images Using Multiview Bootstrapping

BigHand2.2M Benchmark: Hand Pose Dataset and State of the Art Analysis

Real-Time Hand Tracking under Occlusion from an Egocentric RGB-D Sensor

Towards Accurate Multi-person Pose Estimation in the Wild

ArtTrack: Articulated Multi-Person Tracking in the Wild

RMPE: Regional Multi-person Pose Estimation

Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields

PoseTrack: Joint Multi-person Pose Estimation and Tracking

Associative Embedding: End-to-End Learning for Joint Detection and Grouping

Mnemonic Descent Method: A Recurrent Process Applied for End-to-End Face Alignment

DeeperCut: A Deeper, Stronger, and Faster Multi-person Pose Estimation Model

Stacked Hourglass Networks for Human Pose Estimation

Convolutional Pose Machines

Recombinator Networks: Learning Coarse-to-Fine Feature Aggregation

DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimation

Fast and robust hand tracking using detection-guided optimization

Project-Out Cascaded Regression with an application to face alignment

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

Accurate, Robust, and Flexible Real-time Hand Tracking

Adam: A Method for Stochastic Optimization

Real-Time Continuous Pose Recovery of Human Hands Using Convolutional Networks

Multi-scale Deep Learning for Gesture Detection and Localization

Learning Deep Representation for Face Alignment with Auxiliary Attributes

2D Human Pose Estimation: New Benchmark and State of the Art Analysis

Microsoft COCO: Common Objects in Context

300 Faces in-the-Wild Challenge: The First Facial Landmark Localization Challenge

Robust Face Landmark Estimation under Occlusion

Localizing Parts of Faces Using a Consensus of Exemplars

Supervised Descent Method and Its Applications to Face Alignment

Deep Convolutional Network Cascade for Facial Point Detection

Interactive Facial Feature Localization

Face Alignment by Explicit Shape Regression

Face detection, pose estimation, and landmark localization in the wild

Tracking the articulated motion of two strongly interacting hands

Annotated Facial Landmarks in the Wild: A large-scale, real-world database for facial landmark localization

We Are Family: Joint Pose Estimation of Multiple Persons

Multi-PIE

Multi-view Appearance-based 3D Hand Pose Estimation

Overview of the face recognition grand challenge

Diatom autofocusing in brightfield microscopy: a comparative study

Towards Multi-Person Pose Tracking : Bottom-up and Top-down Methods

Mask R-CNN

XM2VTSDB: The Extended M2VTS Database

Field of Study

Journal Information

Name

Volume