ContactPose: A Dataset of Grasps with Object Contact and Hand Pose (2020-07-19T00:00:00.000000Z)

TL;DR

This work introduces ContactPose, the first dataset of hand-object contact paired with hand pose, object pose, and RGB-D images, and uses this data to rigorously evaluate various data representations, heuristics from the literature, and learning methods for contact modeling.

Abstract

Grasping is natural for humans. However, it involves complex hand configurations and soft tissue deformation that can result in complicated regions of contact between the hand and the object. Understanding and modeling this contact can potentially improve hand models, AR/VR experiences, and robotic grasping. Yet, we currently lack datasets of hand-object contact paired with other data modalities, which is crucial for developing and evaluating contact modeling techniques. We introduce ContactPose, the first dataset of hand-object contact paired with hand pose, object pose, and RGB-D images. ContactPose has 2306 unique grasps of 25 household objects grasped with 2 functional intents by 50 participants, and more than 2.9 M RGB-D grasp images. Analysis of ContactPose data reveals interesting relationships between hand pose and contact. We use this data to rigorously evaluate various data representations, heuristics from the literature, and learning methods for contact modeling. Data, code, and trained models are available at this https URL.

Authors

James Hays

15 papers

Samarth Brahmbhatt

2 papers

C. Kemp

6 papers

TL;DR

Abstract

Authors

References69 items

Use the Force, Luke! Learning to Predict Physical Forces by Simulating Effects

FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape From Single RGB Images

Resolving 3D Human Pose Ambiguities With 3D Scene Constraints

Interactive hand pose estimation using a stretch-sensing soft glove

HOnnotate: A Method for 3D Annotation of Hand and Object Poses

Learning the signatures of the human grasp using a scalable tactile glove

ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imaging

Learning Joint Reconstruction of Hands and Manipulated Objects

Expressive Body Capture: 3D Hands, Face, and Body From a Single Image

H+O: Unified Egocentric Recognition of 3D Hand-Object Poses and Interactions

ContactGrasp: Functional Multi-finger Grasp Synthesis from Contact

Fast Graph Representation Learning with PyTorch Geometric

End-to-End Hand Mesh Recovery From a Monocular RGB Image

Learning ambidextrous robot grasping policies

Hand-Object Contact Force Estimation from Markerless Visual Tracking

Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects

Planning Multi-Fingered Grasps as Probabilistic Inference in a Learned Deep Network

A Papier-Mache Approach to Learning 3D Surface Generation

Open3D: A Modern Library for 3D Data Processing

Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies

V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Map

Automatic differentiation in PyTorch

A force and thermal sensing skin for robots in human environments

PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space

Hand Keypoint Detection in Single Images Using Multiview Bootstrapping

First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations

Deep 6-DOF Tracking

Real-Time Joint Tracking of a Hand Manipulating an Object from RGB-D Input

Dex-Net 1.0: A cloud-based network of 3D objects for robust grasp planning using a Multi-Armed Bandit model with correlated rewards

Colorful Image Colorization

The GRASP Taxonomy of Human Grasp Types

A novel type of compliant and underactuated robotic hand for dexterous grasping

Haptic identification of objects using a modular soft robotic gripper

Understanding Everyday Hands in Action from RGB-D Images

VoxNet: A 3D Convolutional Neural Network for real-time object recognition

Hierarchical Density Estimates for Data Clustering, Visualization, and Outlier Detection

Capturing Hands in Action Using Discriminative Salient Points and Physics Simulation

Towards force sensing from vision: Observing hand-object interactions to infer manipulation forces

U-Net: Convolutional Networks for Biomedical Image Segmentation

The Yale human grasping dataset: Grasp, object, and task data in household and machine shop environments

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

Adam: A Method for Stochastic Optimization

3D Shape Reconstruction from 2D Landmarks: A Convex Formulation

Real-Time Continuous Pose Recovery of Human Hands Using Convolutional Networks

Color map optimization for 3D reconstruction with consumer depth cameras

Microsoft COCO: Common Objects in Context

Grasp Frequency and Usage in Daily Household and Machine Shop Tasks

Motion Capture of Hands in Action Using Discriminative Salient Points

Synthesis of detailed hand manipulations using contact sampling

An object-dependent hand pose prior from sparse training data

A sensor fusion approach for recognizing continuous human grasping sequences using hidden Markov models

Graspit! A versatile simulator for robotic grasping

Fast distance queries with rectangular swept sphere volumes

Planning optimal grasps

Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography

Robust Estimation of a Location Parameter

Mask R-CNN

Embodied Hands : Modeling and Capturing Hands and Bodies Together * * Supplementary Material * *

System for detecting and conﬁrming a touch input

Tracking a hand manipulating an object

Author manuscript, published in "Computer Graphics Forum (2005)" Volume xx, Number x Collision Detection for Deformable Objects

Parallel methods for synthesizing whole-hand grasps from generalized prototypes

A heat transfer textbook

A New Algorithm for Unconstrained Optimization

Sensel morph

5th International Workshop on Observing and Understanding Hands in Action

chumpy: Autodiﬀerentiation tool for Python

ContactPose: A Dataset of Grasps with Object Contact and Hand Pose 17

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type