Inferring Super-Resolution Depth from a Moving Light-Source Enhanced RGB-D Sensor: A Variational Approach (2019-12-13T00:00:00.000000Z)

TL;DR

A novel approach towards depth map super-resolution using multi-view uncalibrated photometric stereo with nonconvex variational approach such that no calibration on lighting or camera motion is required due to the formulation of an end-to-end joint optimization problem.

Abstract

A novel approach towards depth map super-resolution using multi-view uncalibrated photometric stereo is presented. Practically, an LED light source is attached to a commodity RGB-D sensor and is used to capture objects from multiple viewpoints with unknown motion. This non-static camera-to-object setup is described with a nonconvex variational approach such that no calibration on lighting or camera motion is required due to the formulation of an end-to-end joint optimization problem. Solving the proposed variational model results in high resolution depth, reflectance and camera pose estimates, as we show on challenging synthetic and real-world datasets.

Authors

D. Cremers

21 papers

Lu Sang

2 papers

Bjoern Haefner

2 papers

TL;DR

Abstract

Authors

References290 items

On the well-posedness of uncalibrated photometric stereo under general lighting

Multi-Frame GAN: Image Enhancement for Stereo Visual Odometry in Low Light

Photometric Segmentation: Simultaneous Photometric Stereo and Masking

Combining Depth Fusion and Photometric Stereo for Fine-Detailed 3D Models

Variational Uncalibrated Photometric Stereo Under General Lighting

One-Day Outdoor Photometric Stereo Using Skylight Estimation

A Differential Volumetric Approach to Multi-View Photometric Stereo

Associative Deep Clustering: Training a Classification Network with No Labels

Photometric Depth Super-Resolution

Robust Fitting of Subdivision Surfaces for Smooth Shape Analysis

The Double Sphere Camera Model

Fight Ill-Posedness with Ill-Posedness: Single-shot Variational Depth Super-Resolution from Shading

Regularized Pointwise Map Recovery from Functional Correspondence

Map-based drone homing using shortcuts

Sequential Convex Programming for Computing Information-Theoretic Minimal Partitions: Nonconvex Nonsmooth Optimization

Detailed Surface Geometry and Albedo Recovery from RGB-D Video under Natural Illumination

A Variational Approach to Shape-from-Shading Under Natural Illumination

Normal Integration: A Survey

Efficient Online Surface Correction for Real-time Large-Scale 3D Reconstruction

Depth Super-Resolution Meets Uncalibrated Photometric Stereo

LED-Based Photometric Stereo: Modeling, Calibration and Numerical Solution

A Non-convex Variational Approach to Photometric Stereo under Inaccurate Lighting

Real-time variational stereo reconstruction with applications to large-scale dense SLAM

Fast odometry and scene flow from RGB-D cameras based on geometric clustering

Multi-view deep learning for consistent semantic mapping with RGB-D cameras

Nonlinear Spectral Image Fusion

Computer Vision für 3-D-Rekonstruktion

Product Manifold Filter: Non-rigid Shape Correspondence via Kernel Density Estimation in the Product Space

Consistent Partial Matching of Shape Collections via Sparse Modeling

Multiframe Motion Coupling for Video Super Resolution

Image-Based Localization Using LSTMs for Structured Feature Correlation

A Combinatorial Solution to Non-Rigid 3D Shape-to-Image Matching

Fast Guided Global Interpolation for Depth and Motion

Depth Map Super-Resolution by Deep Multi-Scale Guidance

From monocular SLAM to autonomous drone exploration

Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

A position free boresight calibration for INS-camera systems

A Deep Primal-Dual Network for Guided Depth Super-Resolution

ATGV-Net: Accurate Depth Super-Resolution

Deep Depth Super-Resolution: Learning Depth Super-Resolution Using Deep Convolutional Neural Network

Non‐Rigid Puzzles

Stream-based Active Learning for efficient and adaptive classification of 3D objects

Matching of Deformable Shapes with Topological Noise

Spectral Decompositions Using One-Homogeneous Functionals

Semi-supervised online learning for efficient classification of objects in 3D data streams

Entropy Minimization for Convex Relaxation Approaches

Dense Continuous-Time Tracking and Mapping with Rolling Shutter RGB-D Cameras

Model-Based Tracking at 300Hz Using Raw Time-of-Flight Observations

Sublabel–Accurate Relaxation of Nonconvex Energies

Reconstructing Street-Scenes in Real-Time from a Driving Car

Motion Cooperation: Smooth Piece-wise Rigid Scene Flow from RGB-D Images

Super-resolution Keyframe Fusion for 3D Modeling with High-Quality Textures

Collaborative Total Variation: A General Framework for Vectorial TV Models

Midrange Geometric Interactions for Semantic Segmentation

Point-wise Map Recovery and Refinement from Functional Correspondence

Efficient minimal-surface regularization of perspective depth maps in variational stereo

RGBD-fusion: Real-time high precision depth recovery

Adopting an unconstrained ray model in light-field cameras for 3D shape reconstruction

Active online confidence boosting for efficient object classification

Image Super-Resolution Using Deep Convolutional Networks

Multi-view Photometric Stereo by Example

Learning Similarities for Rigid and Non-rigid Object Detection

Towards Illumination-Invariant 3D Reconstruction Using ToF RGB-D Cameras

INS-camera calibration without ground control points

Scale-aware navigation of a low-cost quadrocopter with a monocular camera

High-Quality Depth Map Upsampling and Completion for RGB-D Cameras

Environment-Adaptive Learning: How Clustering Helps to Obtain Good Training Data

Learning a Deep Convolutional Network for Image Super-Resolution

Anisotropic Laplace-Beltrami Operators for Shape Analysis

Flow and Color Inpainting for Video Completion

Convex Relaxation of Vectorial Problems with Coupled Regularization

Image Guided Depth Upsampling Using Anisotropic Total Generalized Variation

Proportion Priors for Image Sequence Segmentation

Tree Shape Priors with Connectivity Constraints Using Convex Relaxation on General Graphs

Semi-dense Visual Odometry for a Monocular Camera

Total Variation Regularization for Functions with Values in a Manifold