PVN3D: A Deep Point-Wise 3D Keypoints Voting Network for 6DoF Pose Estimation (2019-11-11T00:00:00.000000Z)

TL;DR

A deep Hough voting network is proposed to detect 3D keypoints of objects and then estimate the 6D pose parameters within a least-squares fitting manner, which is a natural extension of 2D-keypoint approaches that successfully work on RGB based 6DoF estimation.

Abstract

In this work, we present a novel data-driven method for robust 6DoF object pose estimation from a single RGBD image. Unlike previous methods that directly regressing pose parameters, we tackle this challenging task with a keypoint-based approach. Specifically, we propose a deep Hough voting network to detect 3D keypoints of objects and then estimate the 6D pose parameters within a least-squares fitting manner. Our method is a natural extension of 2D-keypoint approaches that successfully work on RGB based 6DoF estimation. It allows us to fully utilize the geometric constraint of rigid objects with the extra depth information and is easy for a network to learn and optimize. Extensive experiments were conducted to demonstrate the effectiveness of 3D-keypoint detection in the 6D pose estimation task. Experimental results also show our method outperforms the state-of-the-art methods by large margins on several benchmarks. Code and video are available at https://github.com/ethnhe/PVN3D.git.

Authors

Yisheng He

2 papers

Wei Sun

2 papers

Haibin Huang

3 papers

TL;DR

Abstract

Authors

References53 items

CDPN: Coordinates-Based Disentangled Pose Network for Real-Time RGB-Based 6-DoF Object Pose Estimation

CullNet: Calibrated and Pose Aware Confidence Scores for Object Pose Estimation

Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation

Deep Hough Voting for 3D Object Detection in Point Clouds

CenterNet: Keypoint Triplets for Object Detection

DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion

Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation

PVNet: Pixel-Wise Voting Network for 6DoF Pose Estimation

Segmentation-Driven 6D Object Pose Estimation

Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects

Implicit 3D Orientation Learning for 6D Object Detection from RGB Images

Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning

Making Deep Heatmaps Robust to Partial Occlusions for 3D Object Pose Estimation

DeepIM: Deep Iterative Matching for 6D Pose Estimation

In Defense of Classical Image Processing: Fast Depth Completion on the CPU

PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation

Real-Time Seamless Single Shot 6D Object Pose Prediction

Frustum PointNets for 3D Object Detection from RGB-D Data

PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes

SSD-6D: Making RGB-Based 3D Detection and 6D Pose Estimation Great Again

Focal Loss for Dense Object Detection

PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space

BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth

Mask R-CNN

6-DoF object pose from semantic keypoints

Global Hypothesis Generation for 6D Object Pose Estimation

Pyramid Scene Parsing Network

Pose Estimation for Augmented Reality: A Hands-On Survey

Multi-view 3D Object Detection Network for Autonomous Driving

Deep Learning of Local RGB-D Patches for 3D Object Detection and 6D Pose Estimation

Stacked Hourglass Networks for Human Pose Estimation

Recovering 6D Object Pose and Predicting Next-Best-View in the Crowd

Deep Residual Learning for Image Recognition

The YCB object and Model set: Towards common benchmarks for manipulation research

PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization

Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views

Viewpoints and keypoints

Single image 3D object detection and pose estimation for grasping

Learning 6D Object Pose Estimation Using 3D Object Coordinates

Model Based Training, Detection and Pose Estimation of Texture-Less 3D Objects in Heavily Cluttered Scenes

Are we ready for autonomous driving? The KITTI vision benchmark suite

Multimodal templates for real-time detection of texture-less objects in heavily cluttered scenes

The MOPED framework: Object recognition and pose estimation for manipulation

Depth-Encoded Hough Voting for Joint Object Detection and Shape Recovery

Discriminative Mixture-of-Templates for Viewpoint Classification

ImageNet: A large-scale hierarchical image database

Viewpoint-independent object class detection using 3D Feature Maps

3D Object Modeling and Recognition Using Local Affine-Invariant Image Descriptors and Multi-View Spatial Constraints

Mean Shift: A Robust Approach Toward Feature Space Analysis

Object recognition from local scale-invariant features

Comparing Images Using the Hausdorff Distance

Least-Squares Fitting of Two 3-D Point Sets

Notes on the OpenSURF Library

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names