Learning to Detect 3D Reflection Symmetry for Single-View Reconstruction (2020-06-17T00:00:00.000000Z)

TL;DR

This work presents a geometry-based end-to-end deep learning framework that first detects the mirror plane of reflection symmetry that commonly exists in man-made objects and then predicts depth maps by finding the intra-image pixel-wise correspondence of the symmetry.

Abstract

3D reconstruction from a single RGB image is a challenging problem in computer vision. Previous methods are usually solely data-driven, which lead to inaccurate 3D shape recovery and limited generalization capability. In this work, we focus on object-level 3D reconstruction and present a geometry-based end-to-end deep learning framework that first detects the mirror plane of reflection symmetry that commonly exists in man-made objects and then predicts depth maps by finding the intra-image pixel-wise correspondence of the symmetry. Our method fully utilizes the geometric cues from symmetry during the test time by building plane-sweep cost volumes, a powerful tool that has been used in multi-view stereopsis. To our knowledge, this is the first work that uses the concept of cost volumes in the setting of single-image 3D reconstruction. We conduct extensive experiments on the ShapeNet dataset and find that our reconstruction method significantly outperforms the previous state-of-the-art single-view 3D reconstruction networks in term of the accuracy of camera poses and depth maps, without requiring objects being completely symmetric. Code is available at this https URL.

Authors

Yichao Zhou

2 papers

Shichen Liu

2 papers

Yi Ma

3 papers

TL;DR

Abstract

Authors

References60 items

Portrait shadow manipulation

Unsupervised Learning of Probably Symmetric Deformable 3D Objects From Images in the Wild

NeurVPS: Neural Vanishing Point Scanning via Conic Convolution

P-MVSNet: Learning Patch-Wise Matching Confidence Aggregation for Multi-View Stereo

MVSCRF: Learning Multi-View Stereo With Conditional Random Fields

Point-Based Multi-View Stereo Network

DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction

What Do Single-View 3D Reconstruction Networks Learn?

DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation

On the Continuity of Rotation Representations in Neural Networks

Occupancy Networks: Learning 3D Reconstruction in Function Space

Unsupervised Learning of Shape and Pose with Differentiable Point Clouds

Learning Shape Priors for Single-View 3D Completion and Reconstruction

Shape Reconstruction Using Volume Sweeping and Learned Photoconsistency

Learning Descriptor, Confidence, and Depth Estimation in Multi-view Stereo

Deep Ordinal Regression Network for Monocular Depth Estimation

MVSNet: Depth Inference for Unstructured Multi-view Stereo

Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images

DeepMVS: Learning Multi-view Stereopsis

Pyramid Stereo Matching Network

A Papier-Mache Approach to Learning 3D Surface Generation

Sparsity Invariant CNNs

SurfaceNet: An End-to-End 3D Neural Network for Multiview Stereopsis

Learning a Multi-View Stereo Machine

Learned Multi-patch Similarity

End-to-End Learning of Geometry and Context for Deep Stereo Regression

A Point Set Generation Network for 3D Object Reconstruction from a Single Image

Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision

Pixelwise View Selection for Unstructured Multi-View Stereo

Just Look at the Image: Viewpoint-Specific Surface Normal Prediction for Improved Multi-View Reconstruction

3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction

Stacked Hourglass Networks for Human Pose Estimation

Deep Residual Learning for Image Recognition

ShapeNet: An Information-Rich 3D Model Repository

Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches

Multi-View Stereo: A Tutorial

Adam: A Method for Stochastic Optimization

Depth Map Prediction from a Single Image using a Multi-Scale Deep Network

Vision meets robotics: The KITTI dataset

Computational Symmetry in Computer Vision and Computer Graphics

Towards Internet-scale multi-view stereo

Measurement of Areas on a Sphere Using Fibonacci and Latitude–Longitude Lattices

Analysis of Building Textures for Reconstructing Partially Occluded Facades

Accurate, Dense, and Robust Multi-View Stereopsis

Detecting Symmetry and Symmetric Constellations of Features

Reconstruction of 3-D Symmetric Curves from Perspective Images without Discrete Features

Handling occlusions in dense multi-view stereo

Detecting rotational symmetries using normalized convolution

Detecting Symmetry in Grey Level Images: The Global Optimization Approach

Symmetry as a Continuous Feature

The importance of symmetry and virtual views in three-dimensional object recognition

Repeated Structures: Image Correspondence Constraints and 3D Structure Recovery

ECE 598: An Invitation to 3-D Vision: From Images to Geometric Models

On group symmetry in multiple view geometry: structure, pose and calibration from single images

An Invitation to 3-D Vision

Multiple view geometry in computer vision. Cambridge university press

Multiple View Geometry in Computer Vision

How is bilateral symmetry of human faces used for recognition of novel views?

On the Detection of the Axes of Symmetry of Symmetric and Almost Symmetric Planar Images

Machine Vision and Applications Efficient Large-scale Multi-view Stereo for Ultra High-resolution Image Sets

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names