ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imaging (2019-04-15T00:00:00.000000Z)

TL;DR

This work presents ContactDB, a novel dataset of contact maps for household objects that captures the rich hand-object contact that occurs during grasping, enabled by use of a thermal camera.

Abstract

Grasping and manipulating objects is an important human skill. Since hand-object contact is fundamental to grasping, capturing it can lead to important insights. However, observing contact through external sensors is challenging because of occlusion and the complexity of the human hand. We present ContactDB, a novel dataset of contact maps for household objects that captures the rich hand-object contact that occurs during grasping, enabled by use of a thermal camera. Participants in our study grasped 3D printed objects with a post-grasp functional intent. ContactDB includes 3750 3D meshes of 50 household objects textured with contact maps and 375K frames of synchronized RGB-D+thermal images. To the best of our knowledge, this is the first large-scale dataset that records detailed contact maps for human grasps. Analysis of this data shows the influence of functional intent and object size on grasping, the tendency to touch/avoid ‘active areas’, and the high frequency of palm and proximal finger contact. Finally, we train state-of-the art image translation and 3D convolution algorithms to predict diverse contact patterns from object shape. Data, code and models are available at https://contactdb.cc.gatech.edu.

Authors

James Hays

15 papers

Samarth Brahmbhatt

2 papers

Cusuh Ham

1 papers

TL;DR

Abstract

Authors

References58 items

Hand-Object Contact Force Estimation from Markerless Visual Tracking

Dealing with Ambiguity in Robotic Grasping via Multiple Predictions

Human grasping database for activities of daily living with depth, color and kinematic data streams

DiverseNet: When One Right Answer is not Enough

Learning Object Grasping for Soft Robot Hands

Open3D: A Modern Library for 3D Data Processing

The complexities of grasping in the wild

Scene Semantic Reconstruction from Egocentric RGB-D-Thermal Videos

BigHand2.2M Benchmark: Hand Pose Dataset and State of the Art Analysis

First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations

Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks

Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics

Unsupervised Image-to-Image Translation Networks

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation

Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypotheses

Image-to-Image Translation with Conditional Adversarial Networks

Real-Time Joint Tracking of a Hand Manipulating an Object from RGB-D Input

A compact representation of human single-object grasping

Tactile mesh saliency

Stochastic Multiple Choice Learning for Training Diverse Deep Ensembles

Soft Robotic Grippers for Biological Sampling on Deep Reefs

Analysis and Observations From the First Amazon Picking Challenge

A novel type of compliant and underactuated robotic hand for dexterous grasping

Understanding Everyday Hands in Action from RGB-D Images

VoxNet: A 3D Convolutional Neural Network for real-time object recognition

Grasp type revisited: A modern perspective on a classical feature for vision

How do we use our hands? Discovering a diverse set of common grasps

Depth-based tracking with physical constraints for robot manipulation

The Yale human grasping dataset: Grasp, object, and task data in household and machine shop environments

Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocols

Grasp planning based on strategy extracted from demonstration

Conditional Generative Adversarial Nets

Color map optimization for 3D reconstruction with consumer depth cameras

Latent Regression Forest: Structured Estimation of 3D Articulated Hand Posture

Deep learning for detecting robotic grasps

Physical Human Interactive Guidance: Identifying Grasping Principles From Human-Planned Grasps

Synthesis of detailed hand manipulations using contact sampling

How Objects Are Grasped: The Interplay between Affordances and End-Goals

Infrared Thermal Imaging: Fundamentals, Research and Applications

3D is here: Point Cloud Library (PCL)

HeatWave: thermal imaging for surface user interaction

An object-dependent hand pose prior from sparse training data

An object for an action, the same object for other actions: effects on hand shaping

Grasp Recognition with Uncalibrated Data Gloves - A Comparison of Classification Methods

The neuroscience of grasping

A sensor fusion approach for recognizing continuous human grasping sequences using hidden Markov models

On the relation between object shape and grasping kinematics.

Simplification and Repair of Polygonal Models Using Volumetric Techniques

A Method for Registration of 3-D Shapes

On grasp choice, grasp models, and the design of hands for manufacturing tasks

Patterns of static prehension in normal hands.

Tactile Logging for Understanding Plausible Tool Use Based on Human Demonstration

GENERATIVE ADVERSARIAL NETS

thermal

ROS: an open-source Robot Operating System

Enough

binvox. http://www.patrickmin.com/ binvox

Clustering by means of medoids

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names