BodyNet: Volumetric Inference of 3D Human Body Shapes (2018-04-13T00:00:00.000000Z)

TL;DR

BodyNet is an end-to-end trainable network that benefits from a volumetric 3D loss, a multi-view re-projection loss, and intermediate supervision of 2D pose, 2D body part segmentation, and 3D pose and achieves state-of-the-art performance.

Abstract

Human shape estimation is an important task for video editing , animation and fashion industry. Predicting 3D human body shape from natural images, however, is highly challenging due to factors such as variation in human bodies, clothing and viewpoint. Prior methods addressing this problem typically attempt to fit parametric body models with certain priors on pose and shape. In this work we argue for an alternative representation and propose BodyNet, a neural network for direct inference of volumetric body shape from a single image. BodyNet is an end-to-end trainable network that benefits from (i) a volumetric 3D loss, (ii) a multi-view re-projection loss, and (iii) intermediate supervision of 2D pose, 2D body part segmentation, and 3D pose. Each of them results in performance improvement as demonstrated by our experiments. To evaluate the method, we fit the SMPL model to our network output and show state-of-the-art results on the SURREAL and Unite the People datasets, outperforming recent approaches. Besides achieving state-of-the-art performance, our method also enables volumetric body-part segmentation.

Authors

Gül Varol

7 papers

I. Laptev

24 papers

Jimei Yang

14 papers

TL;DR

Abstract

Authors

References74 items

Numerical Optimization

2D/3D Pose Estimation and Action Recognition Using Multitask Deep Learning

A Papier-Mache Approach to Learning 3D Surface Generation

PPFNet: Global Context Aware Local Features for Robust 3D Point Matching

DensePose: Dense Human Pose Estimation in the Wild

End-to-End Recovery of Human Shape and Pose

Self-supervised Learning of Motion Capture

Multi-view Dynamic Shape Refinement Using Local Temporal Integration

Towards Accurate Marker-Less Human Shape and Pose Estimation over Time

LCR-Net: Localization-Classification-Regression for Human Pose

Rethinking Reprojection: Closing the Loop for Pose-Aware Shape Reconstruction from a Single Image

A Simple Yet Effective Baseline for 3d Human Pose Estimation

VNect

Multi-view Supervision for Single-View Reconstruction via Differentiable Ray Consistency

Towards 3D Human Pose Estimation in the Wild: A Weakly-Supervised Approach

OctNetFusion: Learning Depth Fusion from Data

Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs

Sparse Inertial Poser: Automatic 3D Human Pose Estimation from Sparse IMUs

Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression

Optical Flow-Based 3D Human Motion Estimation from Monocular Video

Deep Multitask Architecture for Integrated 2D and 3D Human Sensing

Looking beyond appearances: Synthetic training data for deep CNNs in re-identification

Unite the People: Closing the Loop Between 3D and 2D Human Representations

Learning from Synthetic Humans

Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image

DenseReg: Fully Convolutional Dense Shape Regression In-the-Wild

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation

A Point Set Generation Network for 3D Object Reconstruction from a Single Image

Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision

3D Human Pose Estimation from a Single Image via Distance Matrix Regression

Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields

Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose

OctNet: Learning Deep 3D Representations at High Resolutions

Estimation of Human Body Shape in Motion with Wide Clothing

Learning Semantic Deformation Flows with 3D Convolutional Networks

HS-Nets: Estimating Human Body Shape from Silhouettes with Convolutional Neural Networks

Learning Camera Viewpoint Using CNN to Improve 3D Body Pose Estimation

Deep Kinematic Pose Regression

General Automatic Human Shape and Motion Capture Using Volumetric Contour Cues

Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image

MoCap-guided Data Augmentation for 3D Pose Estimation in the Wild

Synthesizing Training Images for Boosting Human 3D Pose Estimation

Learning a Predictable and Generative Vector Representation for Objects

Stacked Hourglass Networks for Human Pose Estimation

Convolutional Pose Machines

Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video

DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimation

Direct Prediction of 3D Body Poses from Motion Compensated Sequences

A Dual-Source Approach for 3D Pose Estimation from a Single Image

VoxNet: A 3D Convolutional Neural Network for real-time object recognition

Pose-conditioned joint angle limits for 3D human pose reconstruction

Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments

Two-Stream Convolutional Networks for Action Recognition in Videos

2D Human Pose Estimation: New Benchmark and State of the Art Analysis

DeepPose: Human Pose Estimation via Deep Neural Networks

ImageNet classification with deep convolutional neural networks

A Naturalistic Open Source Movie for Optical Flow Evaluation

Reconstructing 3D Human Pose from 2D Image Landmarks

Latent structured models for human pose estimation

Estimating human shape and pose from a single image

Detailed Human Shape and Pose from Images

SCAPE: shape completion and animation of people

Simplification and Repair of Polygonal Models Using Volumetric Techniques

Efficient Implementation of Marching Cubes' Cases with Topological Guarantees

Backpropagation Applied to Handwritten Zip Code Recognition

Indirect deep structured learning for 3D human body shape and pose prediction

O-CNN: Octree-based convolutional neural networks for 3D shape analysis

Depth Sweep Regression Forests for Estimating 3D Human Pose from Images

MoSh: Motion and shape capture from sparse markers

Lecture 6.5—RmsProp: Divide the gradient by a running average of its recent magnitude

Clustered Pose and Nonlinear Appearance Models for Human Pose Estimation

LCR-Net: Localization-Classiﬁcation-Regression for Human Pose

Volumetric Inference of 3D Human Body Shapes 17

binvox

Field of Study

Journal Information