SinDDM: A Single Image Denoising Diffusion Model (2022-11-29T00:00:00.000000Z)

TL;DR

This work introduces a framework for training a DDM on a single image, SinDDM, which learns the internal statistics of the training image by using a multi-scale diffusion process, and uses a fully-convolutional light-weight denoiser to drive the reverse diffusion process.

Abstract

Denoising diffusion models (DDMs) have led to staggering performance leaps in image generation, editing and restoration. However, existing DDMs use very large datasets for training. Here, we introduce a framework for training a DDM on a single image. Our method, which we coin SinDDM, learns the internal statistics of the training image by using a multi-scale diffusion process. To drive the reverse diffusion process, we use a fully-convolutional light-weight denoiser, which is conditioned on both the noise level and the scale. This architecture allows generating samples of arbitrary dimensions, in a coarse-to-fine manner. As we illustrate, SinDDM generates diverse high-quality samples, and is applicable in a wide array of tasks, including style transfer and harmonization. Furthermore, it can be easily guided by external supervision. Particularly, we demonstrate text-guided generation from a single image using a pre-trained CLIP model.

Authors

T. Michaeli

5 papers

V. Kulikov

1 papers

S. Yadin

1 papers

TL;DR

Abstract

Authors

References35 items

SinDiffusion: Learning a Diffusion Model From a Single Natural Image

SinFusion: Training Diffusion Models on a Single Image or Video

Imagen Video: High Definition Video Generation with Diffusion Models

Learning to Generate 3D Shapes from a Single Example

An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion

Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding

Any-resolution Training for High-resolution Image Synthesis

Hierarchical Text-Conditional Image Generation with CLIP Latents

Text2LIVE: Text-Driven Layered Image and Video Editing

Generating natural images with direct Patch Distributions Matching

StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets

High-Resolution Image Synthesis with Latent Diffusion Models

GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models

Generative Adversarial Networks

Palette: Image-to-Image Diffusion Models

MUSIQ: Multi-scale Image Quality Transformer

SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations

SinIR: Efficient General Image Manipulation with Single Image Reconstruction

Catch-A-Waveform: Learning to Generate Audio from a Single Short Example

Patchwise Generative ConvNet: Training Energy-Based Models from a Single Natural Image for Internal Learning

Diffusion Models Beat GANs on Image Synthesis

Image Super-Resolution via Iterative Refinement

Drop the GAN: In Defense of Patches Nearest Neighbors as Single Image Generative Models

Learning Transferable Visual Models From Natural Language Supervision

Improved Denoising Diffusion Probabilistic Models

Denoising Diffusion Implicit Models

Hierarchical Patch VAE-GAN: Generating Diverse Videos from a Single Sample

Denoising Diffusion Probabilistic Models

Improved Techniques for Training Single-Image GANs

InGAN: Capturing and Retargeting the “DNA” of a Natural Image

SinGAN: Learning a Generative Model From a Single Natural Image

NIMA: Neural Image Assessment

Deep Unsupervised Learning using Nonequilibrium Thermodynamics

Making a “Completely Blind” Image Quality Analyzer

3. Boundary conditions and the effect of padding Similarly to SinGAN (see Supplementary Sec. 2 in (Shaham et al.

Field of Study

Venue Information

Name

Type

URL

Alternate Names