DensePose: Dense Human Pose Estimation in the Wild (2018-02-01T00:00:00.000000Z)

TL;DR

This work establishes dense correspondences between an RGB image and a surface-based representation of the human body, a task referred to as dense human pose estimation, and improves accuracy through cascading, obtaining a system that delivers highly-accurate results at multiple frames per second on a single gpu.

Abstract

In this work we establish dense correspondences between an RGB image and a surface-based representation of the human body, a task we refer to as dense human pose estimation. We gather dense correspondences for 50K persons appearing in the COCO dataset by introducing an efficient annotation pipeline. We then use our dataset to train CNN-based systems that deliver dense correspondence 'in the wild', namely in the presence of background, occlusions and scale variations. We improve our training set's effectiveness by training an inpainting network that can fill in missing ground truth values and report improvements with respect to the best results that would be achievable in the past. We experiment with fully-convolutional networks and region-based models and observe a superiority of the latter. We further improve accuracy through cascading, obtaining a system that delivers highly-accurate results at multiple frames per second on a single gpu. Supplementary materials, data, code, and videos are provided on the project page http://densepose.org.

Authors

Iasonas Kokkinos

8 papers

R. Güler

4 papers

N. Neverova

6 papers

TL;DR

Abstract

Authors

References51 items

Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies

End-to-End Recovery of Human Shape and Pose

Weakly Supervised Manifold Learning for Dense Semantic Object Correspondence

Benchmarking and Error Diagnosis in Multi-instance Pose Estimation

Unsupervised object learning from dense equivariant image labelling

Mask R-CNN

Look into Person: Self-Supervised Structure-Sensitive Learning and a New Benchmark for Human Parsing

Deep Multitask Architecture for Integrated 2D and 3D Human Sensing

Unite the People: Closing the Loop Between 3D and 2D Human Representations

Learning from Synthetic Humans

Feature Pyramid Networks for Object Detection

DenseReg: Fully Convolutional Dense Shape Regression In-the-Wild

Pyramid Scene Parsing Network

Learning Camera Viewpoint Using CNN to Improve 3D Body Pose Estimation

Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image

MoCap-guided Data Augmentation for 3D Pose Estimation in the Wild

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

DeeperCut: A Deeper, Stronger, and Faster Multi-person Pose Estimation Model

Learning Dense Correspondence via 3D-Guided Cycle Consistency

Synthesizing Training Images for Boosting Human 3D Pose Estimation

Stacked Hourglass Networks for Human Pose Estimation

Convolutional Pose Machines

Hand Pose Estimation through Weakly-Supervised Learning of a Rich Intermediate Representation

Dense Human Body Correspondences Using Convolutional Networks

Metric Regression Forests for Correspondence Estimation

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

Dense Semantic Correspondence Where Every Pixel is a Classifier

Fast R-CNN

Fully convolutional networks for semantic segmentation

Unsupervised Domain Adaptation by Backpropagation

Deeply-Supervised Nets

ImageNet Large Scale Visual Recognition Challenge

Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments

The Pascal Visual Object Classes Challenge: A Retrospective

Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition

Detect What You Can: Detecting and Representing Objects Using Holistic Models and Body Parts

2D Human Pose Estimation: New Benchmark and State of the Art Analysis

Microsoft COCO: Common Objects in Context

Body Parts Dependent Joint Regressors for Human Pose Estimation in Still Images

The Vitruvian manifold: Inferring dense correspondences for one-shot human pose estimation

Articulated people detection and pose estimation: Reshaping the future

Parsing clothing in fashion photographs

Learning people detection models from few training samples

HumanEva: Synchronized Video and Motion Capture Dataset and Baseline Algorithm for Evaluation of Articulated Human Motion

Unsupervised Learning of Object Deformation Models

Data driven image models through continuous joint alignment

Tomas Simon

and Jitendra

and C

Clustered Pose and Nonlinear Appearance Models for Human Pose Estimation

Cmu graphics lab motion capture database

Field of Study

Journal Information

Name

Page