Unsupervised Learning of Shape and Pose with Differentiable Point Clouds (2018-10-22T00:00:00.000000Z)

TL;DR

This work trains a convolutional network to predict both the shape and the pose from a single image by minimizing the reprojection error, and introduces an ensemble of pose predictors which are distill to a single "student" model.

Abstract

We address the problem of learning accurate 3D shape and camera pose from a collection of unlabeled category-specific images. We train a convolutional network to predict both the shape and the pose from a single image by minimizing the reprojection error: given several views of an object, the projections of the predicted shapes to the predicted camera poses should match the provided views. To deal with pose ambiguity, we introduce an ensemble of pose predictors which we then distill to a single "student" model. To allow for efficient learning of high-fidelity shapes, we represent the shapes by point clouds and devise a formulation allowing for differentiable projection of these. Our experiments show that the distilled ensemble of pose predictors learns to estimate the pose accurately, while the point cloud representation allows to predict detailed shape models. The supplementary video can be found at this https URL

Authors

Alexey Dosovitskiy

16 papers

Eldar Insafutdinov

5 papers

Unsupervised Learning of Shape and Pose with Differentiable Point Clouds

TL;DR

Abstract

Authors

References26 items

Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling

3D Interpreter Networks for Viewer-Centered Wireframe Modeling

Multi-view Consistency as Supervisory Signal for Learning Shape and Pose Prediction

FoldingNet: Interpretable Unsupervised Learning on 3D Point Clouds

Neural 3D Mesh Renderer

Photographic Image Synthesis with Cascaded Refinement Networks

Synthesizing 3D Shapes via Modeling Multi-view Depth Maps and Silhouettes with Deep Generative Networks

Learning Efficient Point Cloud Generation for Dense 3D Object Reconstruction

Deformation-driven shape correspondence via shape recognition

Multi-view Supervision for Single-View Reconstruction via Differentiable Ray Consistency

Learning Category-Specific Deformable 3D Models for Object Reconstruction

Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs

A Point Set Generation Network for 3D Object Reconstruction from a Single Image

Learning Shape Abstractions by Assembling Volumetric Primitives

Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision

Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

Unsupervised Learning of 3D Structure from Images

3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction

ShapeNet: An Information-Rich 3D Model Repository

A Versatile Scene Model with Differentiable Visibility Applied to Generative Pose Estimation

Adam: A Method for Stochastic Optimization

OpenDR: An Approximate Differentiable Renderer

Reconstructing PASCAL VOC

Multiple Choice Learning: Learning to Produce Multiple Structured Outputs

Ieee Transactions on Pattern Analysis and Machine Intelligence 1 What Shape Are Dolphins? Building 3d Morphable Models from 2d Images

This Paper Is Included in the Proceedings of the 12th Usenix Symposium on Operating Systems Design and Implementation (osdi '16). Tensorflow: a System for Large-scale Machine Learning Tensorflow: a System for Large-scale Machine Learning

Field of Study

Venue Information

Name

Type

URL

Alternate Names