Guided Depth Map Super-Resolution: A Survey (2023-02-17T00:00:00.000000Z)

TL;DR

A comprehensive survey of recent progress in guided depth map super-resolution (GDSR), which aims to reconstruct a high-resolution depth map from a low-resolution observation with the help of a paired high- resolution color image, is presented.

Abstract

Guided depth map super-resolution (GDSR), which aims to reconstruct a high-resolution depth map from a low-resolution observation with the help of a paired high-resolution color image, is a longstanding and fundamental problem that has attracted considerable attention from computer vision and image processing communities. Myriad novel and effective approaches have been proposed recently, especially with powerful deep learning techniques. This survey is an effort to present a comprehensive survey of recent progress in GDSR. We start by summarizing the problem of GDSR and explaining why it is challenging. Next, we introduce some commonly used datasets and image quality assessment methods. In addition, we roughly classify existing GDSR methods into three categories: filtering-based methods, prior-based methods, and learning-based methods. In each category, we introduce the general description of the published algorithms and design principles, summarize the representative methods, and discuss their highlights and limitations. Moreover, depth-related applications are introduced. Furthermore, we conduct experiments to evaluate the performance of some representative methods based on unified experimental configurations, so as to offer a systematic and fair performance evaluation to readers. Finally, weconclude this survey with possible directions and open problems for further research. All related materials can be found at https://github.com/zhwzhong/Guided-Depth-Map-Super-resolution-A-Survey.

Authors

Xiangyang Ji

9 papers

Debin Zhao

5 papers

Junjun Jiang

3 papers

TL;DR

Abstract

Authors

References174 items

Learning Complementary Correlations for Depth Super-Resolution With Incomplete Data in Real World

CIR-Net: Cross-Modality Interaction and Refinement for RGB-D Salient Object Detection

Bi-Directional Progressive Guidance Network for RGB-D Salient Object Detection

Learning Mutual Modulation for Self-Supervised Cross-Modal Super-Resolution

Learning Implicit Class Knowledge for RGB-D Co-Salient Object Detection With Transformers

Learning Graph Regularisation for Guided Super-Resolution

Depth Map Super-Resolution via Cascaded Transformers Guidance

Weakly Supervised RGB-D Salient Object Detection With Prediction Consistency Training and Active Scribble Boosting

Memory-Augmented Deep Unfolding Network for Guided Image Super-resolution

Revisiting RCAN: Improved Training for Image Super-Resolution

Multi-Modal Convolutional Dictionary Learning

Learning Discriminative Cross-Modality Features for RGB-D Saliency Detection

CODON: On Orchestrating Cross-Domain Attentions for Depth Super-Resolution

CANet: Co-attention network for RGB-D semantic segmentation

Fast, High-Quality Hierarchical Depth-Map Super-Resolution

A Learned Stereo Depth System for Robotic Manipulation in Homes

CU-Net+: Deep Fully Interpretable Network for Multi-Modal Image Restoration

Progressive Coordinate Transforms for Monocular 3D Object Detection

Depth Map Super-Resolution Based on Dual Normal-Depth Regularization and Graph Laplacian Prior

AGO-Net: Association-Guided 3D Point Cloud Object Detection Network

Learning Spatially Variant Linear Representation Models for Joint Filtering

Cross-modality Discrepant Interaction Network for RGB-D Salient Object Detection

BridgeNet: A Joint Learning Network of Depth Map Super-Resolution and Monocular Depth Estimation

Joint Implicit Image Function for Guided Depth Super-Resolution

Depth Super-Resolution by Texture-Depth Transformer

Shape As Points: A Differentiable Poisson Solver

Unsharp Mask Guided Filtering

Deep Convolutional Dictionary Learning for Image Denoising

Segmenter: Transformer for Semantic Segmentation

Weakly Supervised Object Localization and Detection: A Survey

Discrete Cosine Transform Network for Guided Depth Map Super-Resolution

Towards Fast and Accurate Real-World Depth Super-Resolution: Benchmark Dataset and Baseline

High-resolution Depth Maps Imaging via Attention-based Hierarchical Multi-modal Fusion

Learning Scene Structure Guidance via Cross-Task Knowledge Transfer for Single Depth Super-Resolution

PVStereo: Pyramid Voting Module for End-to-End Self-Supervised Stereo Matching

Deep Convolutional Sparse Coding Network For Pansharpening With Guidance Of Side Information

RGB-D Semantic Segmentation and Label-Oriented Voxelgrid Fusion for Accurate 3D Semantic Mapping

EDN: Salient Object Detection via Extremely-Downsampled Network

Weakly Supervised Object Detection Using Proposal- and Semantic-Level Relationships

Learning Continuous Image Representation with Local Implicit Image Function

Depth Map Enhancement by Revisiting Multi-Scale Intensity Guidance Within Coarse-to-Fine Stages

Depth image super-resolution using correlation-controlled color guidance and multi-scale symmetric network

A sparsity-promoting image decomposition model for depth recovery

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Single Pair Cross-Modality Super Resolution

Depth Image Denoising Using Nuclear Norm and Learning Graph Model

RADAR: Robust Algorithm for Depth Image Super Resolution Based on FRI Theory and Multimodal Dictionary Learning

Going From RGB to RGBD Saliency: A Depth-Guided Transformation Model

Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation

Depth upsampling based on deep edge-aware learning

PMBANet: Progressive Multi-Branch Aggregation Network for Scene Depth Super-Resolution

Multi-Direction Dictionary Learning Based Depth Map Super-Resolution With Autoregressive Modeling

Channel Attention Based Iterative Residual Learning for Depth Map Super-Resolution

Color-Guided Depth Image Recovery With Adaptive Data Fidelity and Transferred Graph Laplacian Regularization

Multi-Scale Frequency Reconstruction for Guided Depth Map Super-Resolution via Deep Residual Network

Multimodal Deep Unfolding for Guided Image Super-Resolution

Learned Dynamic Guidance for Depth Image Reconstruction

Deformable Kernel Networks for Joint Image Filtering

Deep Convolutional Neural Network for Multi-Modal Image Restoration and Fusion

DIODE: A Dense Indoor and Outdoor DEpth Dataset

A Generalized Framework for Edge-Preserving and Structure-Preserving Image Smoothing

Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era

Side Window Filtering

Hierarchical Features Driven Residual Learning for Depth Map Super-Resolution

Deep monocular depth estimation leveraging a large-scale outdoor stereo dataset

Salient Object Detection in the Deep Learning Era: An In-Depth Survey

Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks With Octave Convolution

Depth Super-Resolution via Joint Color-Guided Internal and External Regularizations

Simultaneous color-depth super-resolution with conditional generative adversarial networks

Depth Restoration From RGB-D Data via Joint Adaptive Regularization and Thresholding on Manifolds

Deep Color Guided Coarse-to-Fine Convolutional Network Cascade for Depth Image Super-Resolution

A Novel Segmentation Based Depth Map Up-Sampling

Bag of Tricks for Image Classification with Convolutional Neural Networks

PSANet: Point-wise Spatial Attention Network for Scene Parsing

Deep Monocular Depth Estimation via Integration of Global and Local Predictions

Minimum Spanning Forest With Embedded Edge Inconsistency Measurement Model for Guided Depth Map Enhancement