Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wild

Published in

Computer Vision and Pattern Recognition(2020)

External Links:

Generate Graph DownloadPDF

TL;DR

This work introduces a simple and effective network architecture for monocular 3D hand pose estimation consisting of an image encoder followed by a mesh convolutional decoder that is trained through a direct3D hand mesh reconstruction loss.

Abstract

We introduce a simple and effective network architecture for monocular 3D hand pose estimation consisting of an image encoder followed by a mesh convolutional decoder that is trained through a direct 3D hand mesh reconstruction loss. We train our network by gathering a large-scale dataset of hand action in YouTube videos and use it as a source of weak supervision. Our weakly-supervised mesh convolutions-based system largely outperforms state-of-the-art methods, even halving the errors on the in the wild benchmark. The dataset and additional resources are available at https://arielai.com/mesh_hands.

Authors

M. Bronstein

14 papers

S. Zafeiriou

18 papers

Iasonas Kokkinos

8 papers

References63 items

Aligning Latent Spaces for 3D Hand Pose Estimation

SpiralNet++: A Fast and Highly Efficient Mesh Convolution Operator

Variational Object-Aware 3-D Hand Pose From a Single RGB Image

Exploiting Spatial-Temporal Relationships for 3D Pose Estimation via Graph Convolutional Networks

Learning to Reconstruct 3D Human Pose and Shape via Model-Fitting in the Loop

Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wild

Published in

Computer Vision and Pattern Recognition(2020)

External Links:

Generate Graph DownloadPDF

TL;DR

Abstract

Authors

M. Bronstein

14 papers

S. Zafeiriou

18 papers

Iasonas Kokkinos

8 papers

References63 items

Aligning Latent Spaces for 3D Hand Pose Estimation

SpiralNet++: A Fast and Highly Efficient Mesh Convolution Operator

Variational Object-Aware 3-D Hand Pose From a Single RGB Image

Exploiting Spatial-Temporal Relationships for 3D Pose Estimation via Graph Convolutional Networks

Learning to Reconstruct 3D Human Pose and Shape via Model-Fitting in the Loop

Dominik Kulon

1 papers

FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape From Single RGB Images

HoloPose: Holistic 3D Human Reconstruction In-The-Wild

Neural 3D Morphable Models: Spiral Convolutional Networks for 3D Shape Representation Learning and Generation

Convolutional Mesh Regression for Single-Image Human Shape Reconstruction

Single Image 3D Hand Reconstruction with Mesh Convolutions

Learning Joint Reconstruction of Hands and Manipulated Objects

Expressive Body Capture: 3D Hands, Face, and Body From a Single Image

H+O: Unified Egocentric Recognition of 3D Hand-Object Poses and Interactions

Pushing the Envelope for RGB-Based Dense 3D Hand Pose Estimation via Neural Rendering

3D Hand Shape and Pose Estimation From a Single RGB Image

End-to-End Hand Mesh Recovery From a Monocular RGB Image

3D Hand Shape and Pose From Images in the Wild

Feature Boosting Network For 3D Pose Estimation

Monocular Total Capture: Posing Face, Body, and Hands in the Wild

Disentangling Latent Hands for Image Synthesis and Pose Estimation

A Simple Approach to Intrinsic Correspondence Learning on Unstructured 3D Meshes

Weakly-Supervised 3D Hand Pose Estimation from Monocular RGB Images

Neural Body Fitting: Unifying Deep Learning and Model Based Human Pose and Shape Estimation

Generating 3D faces using Convolutional Mesh Autoencoders

Monocular RGB Hand Pose Inference from Unsupervised Refinable Nets

Learning to Estimate 3D Human Pose and Shape from a Single Color Image

BodyNet: Volumetric Inference of 3D Human Body Shapes

Cross-Modal Deep Variational Hand Pose Estimation

Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies

End-to-End Recovery of Human Shape and Pose

Using a Single RGB Frame for Real Time 3D Hand Pose Estimation in the Wild

3D Hand Pose Estimation: From Current Achievements to Future Goals

GANerated Hands for Real-Time 3D Hand Tracking from Monocular RGB

Graph Attention Networks

DeepPrior++: Improving Fast and Accurate 3D Hand Pose Estimation

FeaStNet: Feature-Steered Graph Convolutions for 3D Shape Analysis

CayleyNets: Graph Convolutional Neural Networks With Complex Rational Spectral Filters

A hand pose tracking benchmark from stereo matching

Learning to Estimate 3D Hand Pose from Single RGB Images

Hand Keypoint Detection in Single Images Using Multiview Bootstrapping

Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks

Unite the People: Closing the Loop Between 3D and 2D Human Representations

Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs

Geometric Deep Learning: Going beyond Euclidean data

Semi-Supervised Classification with Graph Convolutional Networks

Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image

Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering

Robust 3D Hand Pose Estimation in Single Depth Images: From Single-View CNN to Multi-View CNNs

Learning shape correspondence with anisotropic convolutional neural networks

Convolutional Pose Machines

Rule of thumb: Deep derotation for improved fingertip detection

Depth-Based Hand Pose Estimation: Methods, Data, and Challenges

Geodesic Convolutional Neural Networks on Riemannian Manifolds

2D Human Pose Estimation: New Benchmark and State of the Art Analysis

Microsoft COCO: Common Objects in Context

Spectral Networks and Locally Connected Networks on Graphs

Surface simplification using quadric error metrics

Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks

Surface Simpliﬁcation Using Quadric Error Metrics

Embodied Hands : Modeling and Capturing Hands and Bodies Together * * Supplementary Material * *

Indirect deep structured learning for 3D human body shape and pose prediction

Tanks and temples: Benchmarking large-scale scene reconstruction

Nz sign language exercises

Field of Study

Computer Science

Journal Information

Name

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Page

11851-11859

Venue Information

Name

Computer Vision and Pattern Recognition

Type

conference

URL

https://ieeexplore.ieee.org/xpl/conhome.jsp?punumber=1000147

Alternate Names

CVPR
Comput Vis Pattern Recognit