Pose from Shape: Deep Pose Estimation for Arbitrary 3D Objects (2019-06-12T00:00:00.000000Z)

TL;DR

This work proposes a completely generic deep pose estimation approach, which does not require the network to have been trained on relevant categories, nor objects in a category to have a canonical pose, and demonstrates that this method boosts performances for supervised category pose estimation on standard benchmarks.

Abstract

Most deep pose estimation methods need to be trained for specific object instances or categories. In this work we propose a completely generic deep pose estimation approach, which does not require the network to have been trained on relevant categories, nor objects in a category to have a canonical pose. We believe this is a crucial step to design robotic systems that can interact with new objects in the wild not belonging to a predefined category. Our main insight is to dynamically condition pose estimation with a representation of the 3D shape of the target object. More precisely, we train a Convolutional Neural Network that takes as input both a test image and a 3D model, and outputs the relative 3D pose of the object in the input image with respect to the 3D model. We demonstrate that our method boosts performances for supervised category pose estimation on standard benchmarks, namely Pascal3D+, ObjectNet3D and Pix3D, on which we provide results superior to the state of the art. More importantly, we show that our network trained on everyday man-made objects from ShapeNet generalizes without any additional training to completely new types of 3D objects by providing results on the LINEMOD dataset as well as on natural entities such as animals from ImageNet.

Authors

Renaud Marlet

5 papers

Mathieu Aubry

9 papers

Yang Xiao

8 papers

TL;DR

Abstract

Authors

References59 items

Matching RGB Images to CAD Models for Object Pose Estimation

Deep Model-Based 6D Pose Refinement in RGB

A Mixed Classification-Regression Framework for 3D Pose Estimation from 2D Images

Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling

Making Deep Heatmaps Robust to Partial Occlusions for 3D Object Pose Estimation

DeepIM: Deep Iterative Matching for 6D Pose Estimation

3D Pose Estimation and 3D Model Retrieval for Objects in the Wild

StarMap for Category-Agnostic Keypoint and Viewpoint Estimation

A Unified Framework for Multi-View Multi-Class Object Pose Estimation

A Papier-Mache Approach to Learning 3D Surface Generation

Dynamic Graph CNN for Learning on Point Clouds

PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation

Real-Time Seamless Single Shot 6D Object Pose Prediction

Frustum PointNets for 3D Object Detection from RGB-D Data

PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes

SSD-6D: Making RGB-Based 3D Detection and 6D Pose Estimation Great Again

Exploring Spatial Context for 3D Semantic Segmentation of Point Clouds

Real-Time Monocular Pose Estimation of 3D Objects Using Temporally Consistent Local Color Histograms

PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space

BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth

6-DoF object pose from semantic keypoints

Geometric Loss Functions for Camera Pose Regression with Deep Learning

DenseReg: Fully Convolutional Dense Shape Regression In-the-Wild

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation

3D Bounding Box Estimation Using Deep Learning and Geometry

ObjectNet3D: A Large Scale Database for 3D Object Recognition

Crafting a multi-task CNN for viewpoint estimation

A Comparative Analysis and Study of Multiview CNN Models for Joint Object Categorization and Pose Estimation

Uncertainty-Driven 6D Pose Estimation of Objects and Scenes from a Single RGB Image

Deep Residual Learning for Image Recognition

ShapeNet: An Information-Rich 3D Model Repository

PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization

Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views

Multi-view Convolutional Neural Networks for 3D Shape Recognition

Adam: A Method for Stochastic Optimization

Viewpoints and keypoints

Very Fast Solution to the PnP Problem with Algebraic Outlier Rejection

Beyond PASCAL: A benchmark for 3D object detection in the wild

Revisiting the PnP Problem: A Fast, General and Optimal Solution

Model Based Training, Detection and Pose Estimation of Texture-Less 3D Objects in Heavily Cluttered Scenes

A Robust O(n) Solution to the Perspective-n-Point Problem

Are we ready for autonomous driving? The KITTI vision benchmark suite

Deformable registration using edge‐preserving scale space for adaptive image‐guided radiation therapy

SUN database: Large-scale scene recognition from abbey to zoo

DAISY: An Efficient Dense Descriptor Applied to Wide-Baseline Stereo

ImageNet: A large-scale hierarchical image database

Synergistic Face Detection and Pose Estimation with Energy-Based Models

Distinctive Image Features from Scale-Invariant Keypoints

Fitting Parameterized Three-Dimensional Models to Images

Robust Estimation of a Location Parameter

Deep Learning on Point Sets for 3 D Classification and Segmentation

Improving Object Classification using Pose Information

The PAS- CAL Visual Object Classes Challenge 2012 (VOC2012) Results

Distinctive Image Features from Scale-Invariant Keypoints

On robust estimation of the location parameter

Ieee Transactions on Pattern Analysis and Machine Intelligence 1 Gradient Response Maps for Real-time Detection of Texture-less Objects

Noname manuscript No. (will be inserted by the editor) EPnP: An Accurate O(n) Solution to the PnP Problem

Free3D

DEEP POSE ESTIMATION FOR ARBITRARY 3D OBJECTS

Field of Study

Venue Information

Name

Type

URL

Alternate Names