Robust, Occlusion-aware Pose Estimation for Objects Grasped by Adaptive Hands (2020-03-07T00:00:00.000000Z)

TL;DR

A depth-based framework for robust pose estimation and short response times for adaptive hands, which demonstrates the accuracy and computational efficiency of the framework when applied on challenging, highly-occluded scenarios for different object types.

Abstract

Many manipulation tasks, such as placement or within-hand manipulation, require the object's pose relative to a robot hand. The task is difficult when the hand significantly occludes the object. It is especially hard for adaptive hands, for which it is not easy to detect the finger's configuration. In addition, RGB-only approaches face issues with texture-less objects or when the hand and the object look similar. This paper presents a depth-based framework, which aims for robust pose estimation and short response times. The approach detects the adaptive hand's state via efficient parallel search given the highest overlap between the hand's model and the point cloud. The hand's point cloud is pruned and robust global registration is performed to generate object pose hypotheses, which are clustered. False hypotheses are pruned via physical reasoning. The remaining poses' quality is evaluated given agreement with observed data. Extensive evaluation on synthetic and real data demonstrates the accuracy and computational efficiency of the framework when applied on challenging, highly-occluded scenarios for different object types. An ablation study identifies how the framework's components help in performance. This work also provides a dataset for in-hand 6D object pose estimation. Code and dataset are available at: https://github.com/wenbowen123/icra20-hand-object-pose

Authors

Bowen Wen

6 papers

Chaitanya Mitash

3 papers

Kostas E. Bekris

8 papers

TL;DR

Abstract

Authors

References62 items

Scene-level Pose Estimation for Multiple Instances of Densely Packed Objects

PoseRBPF: A Rao–Blackwellized Particle Filter for 6-D Object Pose Tracking

Learning Joint Reconstruction of Hands and Manipulated Objects

Manipulation by Feel: Touch-Based Control with Deep Predictive Models

Learning to Estimate Pose and Shape of Hand-Held Objects from RGB Images

Learning a State Transition Model of an Underactuated Adaptive Hand

DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion

A modular framework for model-based visual tracking using edge, texture and depth features

RGB-D object detection and semantic segmentation for autonomous manipulation in clutter

Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects

Implicit 3D Orientation Learning for 6D Object Detection from RGB Images

Learning dexterous in-hand manipulation

Learning Dexterous In-Hand Manipulation.

Fusing Joint Measurements and Visual Features for In-Hand Object Pose Estimation

Physics-based scene-level reasoning for object pose estimation in clutter

Principal Components of Touch

Robust 6D Object Pose Estimation with Stochastic Congruent Sets

Learning Modes of Within-Hand Manipulation

Dexterous Manipulation Graphs

Real-Time Seamless Single Shot 6D Object Pose Prediction

PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes

Improving 6D Pose Estimation of Objects in Clutter Via Physics-Aware Monte Carlo Tree Search

Realtime State Estimation with Tactile and Visual Sensing. Application to Planar Manipulation

Rotational Subgroup Voting and Pose Clustering for Robust 3D Object Recognition

T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-Less Objects

Probabilistic Articulated Real-Time Tracking for Robot Manipulation

Fast Global Registration

Using Vision for Pre- and Post-grasping Object Localization for Soft Hands

Multi-view self-supervised deep learning for 6D pose estimation in the Amazon Picking Challenge

3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions

Depth-based object tracking using a Robust Gaussian Filter

Analysis and Observations From the First Amazon Picking Challenge

In-Hand Object Pose Estimation Using Covariance-Based Tactile To Geometry Matching

Haptic identification of objects using a modular soft robotic gripper

The YCB object and Model set: Towards common benchmarks for manipulation research

You Only Look Once: Unified, Real-Time Object Detection

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

Depth-based tracking with physical constraints for robot manipulation

Single image 3D object detection and pose estimation for grasping

DART: Dense Articulated Real-Time Tracking

Realtime and Robust Hand Tracking from Depth

Combining touch and vision for the estimation of an object's pose during manipulation

RGB-D object tracking: A particle filter approach on GPU

Online in-hand object localization

Probabilistic object tracking using a range camera

Open-Loop Precision Grasping With Underactuated Hands Inspired by a Human Manipulation Strategy

Model Based Training, Detection and Pose Estimation of Texture-Less 3D Objects in Heavily Cluttered Scenes

Real-Time Human Pose Tracking from Range Data

Technical Demonstration on Model Based Training, Detection and Pose Estimation of Texture-Less 3D Objects in Heavily Cluttered Scenes

Efficient scene simulation for robust monte carlo localization using an RGB-D camera

The application of particle filtering to grasping acquisition with visual occlusion and tactile sensing

Fast object localization and pose estimation in heavy clutter for robotic bin picking

Autonomous manipulation with a general-purpose simple hand

Fusion of stereo vision, force-torque, and joint sensors for estimation of in-hand object location

Model globally, match locally: Efficient and robust 3D object recognition

Object modeling by registration of multiple range images

Integrating Vision and Touch for Object Recognition Tasks

Super4PCS: Fast Global Pointcloud Registration via Smart Indexing

Belief-Space Planning Using Learned Models with Application to Underactuated Hands

Aruco: a minimal library for augmented reality applications based on opencv

4-points congruent sets for robust surface registration

Model globally , match locally : Efficient and robust 3 d object recognition , ” in 2010 IEEE computer society conference on computer vision and pattern recognition

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names