Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model (2022-12-01T00:00:00.000000Z)

TL;DR

This work proposes the Denoising Diffusion Null-Space Model (DDNM), a novel zero-shot framework for arbitrary linear IR problems, including but not limited to image super-resolution, colorization, inpainting, compressed sensing, and deblurring, and proposes an enhanced and robust version, dubbed DDNM+, to support noisy restoration and improve restoration quality for hard tasks.

Abstract

Most existing Image Restoration (IR) models are task-specific, which can not be generalized to different degradation operators. In this work, we propose the Denoising Diffusion Null-Space Model (DDNM), a novel zero-shot framework for arbitrary linear IR problems, including but not limited to image super-resolution, colorization, inpainting, compressed sensing, and deblurring. DDNM only needs a pre-trained off-the-shelf diffusion model as the generative prior, without any extra training or network modifications. By refining only the null-space contents during the reverse diffusion process, we can yield diverse results satisfying both data consistency and realness. We further propose an enhanced and robust version, dubbed DDNM+, to support noisy restoration and improve restoration quality for hard tasks. Our experiments on several IR tasks reveal that DDNM outperforms other state-of-the-art zero-shot IR methods. We also demonstrate that DDNM+ can solve complex real-world applications, e.g., old photo restoration.

Authors

Yinhuai Wang

1 papers

Jiwen Yu

1 papers

Jian Zhang

1 papers

TL;DR

Abstract

Authors

References44 items

GAN Prior based Null-Space Learning for Consistent Super-Resolution

Diffusion Posterior Sampling for General Noisy Inverse Problems

Improving Diffusion Models for Inverse Problems using Manifold Constraints

Video Diffusion Models

Panini-Net: GAN Prior Based Degradation-Aware Feature Interpolation for Face Restoration

Denoising Diffusion Restoration Models

RePaint: Inpainting using Denoising Diffusion Probabilistic Models

Generative Adversarial Networks

Deblurring via Stochastic Refinement

Solving Inverse Problems in Medical Imaging with Score-Based Generative Models

Palette: Image-to-Image Diffusion Models

SwinIR: Image Restoration Using Swin Transformer

ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models

Alias-Free Generative Adversarial Networks

GAN Prior Embedded Network for Blind Face Restoration in the Wild

Diffusion Models Beat GANs on Image Synthesis

Image Super-Resolution via Iterative Refinement

Designing a Practical Degradation Model for Deep Blind Image Super-Resolution

Towards Real-World Blind Face Restoration with Generative Facial Prior

GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution

Score-Based Generative Modeling through Stochastic Differential Equations

Denoising Diffusion Implicit Models

Denoising Diffusion Probabilistic Models

Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation

Deep Unfolding Network for Image Super-Resolution

PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models

Explorable Super Resolution

Analyzing and Improving the Image Quality of StyleGAN

Deep Decomposition Learning for Inverse Imaging Problems

Generative Modeling by Estimating Gradients of the Data Distribution

A Style-Based Generator Architecture for Generative Adversarial Networks

Deep null space learning for inverse problems: convergence analysis and rates

Neural Discrete Representation Learning

NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study

GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium

ISTA-Net: Interpretable Optimization-Inspired Deep Network for Image Compressive Sensing

Image Super-Resolution Using Deep Convolutional Networks

Auto-Encoding Variational Bayes

2022) solves noise-free image inpainting problems, where n = 0 and A represents the mask operation. RePaint first create a noised version of the masked image y

2022b;a) resort to such optimization, e.g., DPS (Chung et al., 2022a) uses

DDNM WITH RECENT DIFFUSION-BASED IR METHODS Here we provide detailed comparison between DDNM and recent diffusion-based IR methods, including RePaint (Lugmayr et al., 2022)

2020) propose a conditional sampling strategy in diffusion

DIGITAL IMAGE RESTORATION

Bayesian-Based Iterative Method of Image Restoration

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names