Multi-Stage Progressive Image Restoration (2021-02-04T00:00:00.000000Z)

TL;DR

This paper proposes a novel synergistic design that can optimally balance these competing goals in image restoration tasks, that progressively learns restoration functions for the degraded inputs, thereby breaking down the overall recovery process into more manageable steps.

Abstract

Image restoration tasks demand a complex balance between spatial details and high-level contextualized information while recovering images. In this paper, we propose a novel synergistic design that can optimally balance these competing goals. Our main proposal is a multi-stage architecture, that progressively learns restoration functions for the degraded inputs, thereby breaking down the overall recovery process into more manageable steps. Specifically, our model first learns the contextualized features using encoder-decoder architectures and later combines them with a high-resolution branch that retains local information. At each stage, we introduce a novel per-pixel adaptive design that leverages in-situ supervised attention to reweight the local features. A key ingredient in such a multi-stage architecture is the information exchange between different stages. To this end, we propose a two-faceted approach where the information is not only exchanged sequentially from early to late stages, but lateral connections between feature processing blocks also exist to avoid any loss of information. The resulting tightly interlinked multi-stage architecture, named as MPRNet, delivers strong performance gains on ten datasets across a range of tasks including image deraining, deblurring, and denoising. The source code and pre-trained models are available at https://github.com/swz30/MPRNet.

Authors

Ling Shao

32 papers

Salman Hameed Khan

14 papers

Munawar Hayat

12 papers

TL;DR

Abstract

Authors

References99 items

Real-World Blur Dataset for Learning and Benchmarking Deblurring Algorithms

Dual Adversarial Network: Toward Real-world Noise Removal and Noise Generation

Spatially-Attentive Patch-Hierarchical Network for Adaptive Motion Deblurring

Deblurring by Realistic Blurring

Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation

Multi-Scale Progressive Fusion Network for Single Image Deraining

CycleISP: Real Image Restoration via Improved Data Synthesis

Learning Enriched Features for Real Image Restoration and Enhancement

Transfer Learning From Synthetic to Real-Noise Denoising With Adaptive Instance Normalization

Spatial-Adaptive Network for Single Image Denoising

Deep Learning on Image Denoising: An overview

Multi-Temporal Recurrent Neural Networks For Progressive Non-Uniform Single Image Deblurring With Incremental Temporal Training

Human-Aware Motion Deblurring

Variational Denoising Network: Toward Blind Noise Modeling and Removal

SPGNet: Semantic Prediction Guidance for Scene Parsing

DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better

Densely Residual Laplacian Super-Resolution

Dynamic Scene Deblurring With Parameter Selective Sharing and Nested Skip Connections

Uncertainty Guided Multi-Scale Residual Learning-Using a Cycle Spinning CNN for Single Image De-Raining

Second-Order Attention Network for Single Image Super-Resolution

NTIRE 2019 Challenge on Real Image Denoising: Methods and Results

A Deep Journey into Super-resolution

Real Image Denoising With Feature Attention

Deep Stacked Hierarchical Multi-Patch Network for Image Deblurring

Region-Adaptive Dense Network for Efficient Motion Deblurring

MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation

Residual Non-local Attention Networks for Image Restoration

Progressive Image Deraining Networks: A Better and Simpler Baseline

Rethinking on Multi-Stage Networks for Human Pose Estimation

Residual Dense Network for Image Restoration

CCNet: Criss-Cross Attention for Semantic Segmentation

Unprocessing Images for Learned Raw Denoising

Stacked Spatio-Temporal Graph Convolutional Networks for Action Segmentation

Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks

Dual Attention Network for Scene Segmentation

PSANet: Point-wise Spatial Attention Network for Scene Parsing

ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks

Semi-Supervised Transfer Learning for Image Rain Removal

CBAM: Convolutional Block Attention Module

Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining

Toward Convolutional Blind Denoising of Real Photographs

Image Super-Resolution Using Very Deep Residual Channel Attention Networks

Dynamic Scene Deblurring Using Spatially Variant Recurrent Neural Networks

A High-Quality Denoising Dataset for Smartphone Cameras

Lightweight Pyramid Networks for Image Deraining

Learning to See in the Dark

Density-Aware Single Image De-raining Using a Multi-stream Dense Network

Scale-Recurrent Network for Deep Image Deblurring

Attentive Generative Adversarial Network for Raindrop Removal from A Single Image

Non-local Neural Networks

Cascaded Pyramid Network for Multi-person Pose Estimation

DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks

Squeeze-and-Excitation Networks

Benchmarking Denoising Algorithms with Real Photographs

Removing Rain from Single Images via a Deep Detail Network

Learning Deep CNN Denoiser Prior for Image Restoration

Image De-Raining Using a Conditional Generative Adversarial Network

From Motion Blur to Motion Flow: A Deep Learning Solution for Removing Heterogeneous Motion Blur

Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring

Deconvolution and Checkerboard Artifacts

Deep Joint Rain Detection and Removal from a Single Image

Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

Clearing the Skies: A Deep Network Architecture for Single-Image Rain Removal

Densely Connected Convolutional Networks

Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising

SGDR: Stochastic Gradient Descent with Warm Restarts

Blind Image Deblurring Using Dark Channel Prior

Rain Streak Removal Using Layer Priors

Stacked Hourglass Networks for Human Pose Estimation

Deep Residual Learning for Image Recognition

Removing Rain from a Single Image via Discriminative Sparse Coding

Multi-Scale Context Aggregation by Dilated Convolutions

ParseNet: Looking Wider to See Better

U-Net: Convolutional Networks for Biomedical Image Segmentation

Image Super-Resolution Using Deep Convolutional Networks

Adam: A Method for Stochastic Optimization