Restoring Images with Unknown Degradation Factors by Recurrent Use of a Multi-branch Network (2019-07-10T00:00:00.000000Z)

TL;DR

This paper designs a network having multiple pairs of input and output branches and uses it in a recurrent fashion such that a different branch pair is used at each of the recurrent paths and proposes a two-step training method for the network, which consists of multi-task learning and finetuning.

Abstract

The employment of convolutional neural networks has achieved unprecedented performance in the task of image restoration for a variety of degradation factors. However, high-performance networks have been specifically designed for a single degradation factor. In this paper, we tackle a harder problem, restoring a clean image from its degraded version with an unknown degradation factor, subject to the condition that it is one of the known factors. Toward this end, we design a network having multiple pairs of input and output branches and use it in a recurrent fashion such that a different branch pair is used at each of the recurrent paths. We reinforce the shared part of the network with improved components so that it can handle different degradation factors. We also propose a two-step training method for the network, which consists of multi-task learning and finetuning. The experimental results show that the proposed network yields at least comparable or sometimes even better performance on four degradation factors as compared with the best dedicated network for each of the four. We also test it on a further harder task where the input image contains multiple degradation factors that are mixed with unknown mixture ratios, showing that it achieves better performance than the previous state-of-the-art method designed for the task.

Authors

Xing Liu

1 papers

M. Suganuma

1 papers

Xiyang Luo

1 papers

TL;DR

Abstract

Authors

References72 items

Generative Adversarial Networks

Dual Residual Networks Leveraging the Potential of Paired Operations for Image Restoration

Attention-Based Adaptive Selection of Operations for Image Restoration in the Presence of Unknown Combined Distortions

Multi-Task Learning of Hierarchical Vision-Language Representation

Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks

Deep Multi-task Learning to Recognise Subtle Facial Expressions of Mental States

Non-locally Enhanced Encoder-Decoder Network for Single Image De-raining

Competitive Inner-Imaging Squeeze and Excitation for Residual Network

CBAM: Convolutional Block Attention Module

A Trilateral Weighted Sparse Coding Scheme for Real-World Image Denoising

Image Super-Resolution Using Very Deep Residual Channel Attention Networks

Multi-task Adversarial Network for Disentangled Feature Learning

Image Restoration by Estimating Frequency Distribution of Local Patches

PAD-Net: Multi-tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsing

Crafting a Toolchain for Image Restoration by Deep Reinforcement Learning

Gated Fusion Network for Single Image Dehazing

Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps With Accurate Object Boundaries

Densely Connected Pyramid Dehazing Network

Exploiting the Potential of Standard Convolutional Autoencoders for Image Restoration by Evolutionary Search

Tell Me Where to Look: Guided Attention Inference Network

Scale-Recurrent Network for Deep Image Deblurring

RESIDE: A Benchmark for Single Image Dehazing

Attentive Generative Adversarial Network for Raindrop Removal from A Single Image

DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks

xUnit: Learning a Spatial Activation Function for Efficient Image Restoration

Learning to Super-Resolve Blurry Face and Text Images

Squeeze-and-Excitation Networks

Learning Blind Motion Deblurring

Weakly Supervised Actor-Action Segmentation via Robust Multi-task Ranking

Enhanced Deep Residual Networks for Single Image Super-Resolution

NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results

Removing Rain from Single Images via a Deep Detail Network

Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics

Air-light estimation using haze-lines

Residual Attention Network for Image Classification

Deep Generative Adversarial Compression Artifact Removal

Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring

Single Image Dehazing via Multi-scale Convolutional Neural Networks

Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising

Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network

Non-local Image Dehazing

DehazeNet: An End-to-End System for Single Image Haze Removal

D3: Deep Dual-Domain Based Fast Restoration of JPEG-Compressed Images

Patch Group Based Nonlocal Self-Similarity Prior Learning for Image Denoising

External Patch Prior Guided Internal Clustering for Image Denoising

Rain drop detection and removal using K-Means clustering

Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration

Compression Artifacts Reduction by a Deep Convolutional Network

Learning a convolutional neural network for non-uniform motion blur removal

Fast Image Restoration for Spatially Varying Defocus Blur of Imaging Sensor

Deblurring Text Images via L0-Regularized Intensity and Gradient Prior

Reducing Artifacts in JPEG Decompression Via a Learned Dictionary

A Generalized Low-Rank Appearance Model for Spatio-temporally Correlated Rain Streaks

Efficient Image Dehazing with Boundary Constraint and Contextual Regularization

Unnatural L0 Sparse Representation for Natural Image Deblurring

Bayesian Blind Deconvolution with General Sparse Image Priors

Automatic Single-Image-Based Rain Streaks Removal via Image Decomposition

Local MAP estimation for quality improvement of compressed color images

Two-Phase Kernel Estimation for Robust Motion Deblurring

Single image haze removal using dark channel prior

Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering

Removing camera shake from a single photograph

Fields of Experts: a framework for learning image priors

Rainy weather recognition from in-vehicle camera images for driver assistance

Multitask Learning

Regularized reconstruction to reduce blocking artifacts of block discrete cosine transform compressed images

Nonlinear total variation based noise removal algorithms

Joint denoising and decompression using CNN regularization

A Total Variation-Based JPEG Decompression Model

Visualizing Data using t-SNE

Ensemble Learning for Blind Image Separation and Deconvolution

To appear in the ACM SIGGRAPH conference proceedings High-quality Motion Deblurring from a Single Image ∗

Field of Study

Journal Information

Name