Diffusion Models Beat GANs on Image Synthesis (2021-05-11T00:00:00.000000Z)

TL;DR

It is shown that diffusion models can achieve image sample quality superior to the current state-of-the-art generative models, and classifier guidance combines well with upsampling diffusion models, further improving FID to 3.94 on ImageNet 256$\times$256 and 3.85 on imageNet 512$\ times$512.

Abstract

We show that diffusion models can achieve image sample quality superior to the current state-of-the-art generative models. We achieve this on unconditional image synthesis by finding a better architecture through a series of ablations. For conditional image synthesis, we further improve sample quality with classifier guidance: a simple, compute-efficient method for trading off diversity for fidelity using gradients from a classifier. We achieve an FID of 2.97 on ImageNet 128$\times$128, 4.59 on ImageNet 256$\times$256, and 7.72 on ImageNet 512$\times$512, and we match BigGAN-deep even with as few as 25 forward passes per sample, all while maintaining better coverage of the distribution. Finally, we find that classifier guidance combines well with upsampling diffusion models, further improving FID to 3.94 on ImageNet 256$\times$256 and 3.85 on ImageNet 512$\times$512. We release our code at https://github.com/openai/guided-diffusion

Authors

Alex Nichol

6 papers

Prafulla Dhariwal

8 papers

TL;DR

Abstract

Authors

References81 items

Generative Adversarial Networks

On Buggy Resizing Libraries and Surprising Subtleties in FID Calculation

Image Super-Resolution via Iterative Refinement

StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery

Generating Images with Sparse Representations

Learning Transferable Visual Models From Natural Language Supervision

Zero-Shot Text-to-Image Generation

Improved Denoising Diffusion Probabilistic Models

Generating images from caption and vice versa via CLIP-Guided Generative Latent Space Search

Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed

Taming Transformers for High-Resolution Image Synthesis

Learning Energy-Based Models by Diffusion Recovery Likelihood

Score-Based Generative Modeling through Stochastic Differential Equations

Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images

Denoising Diffusion Implicit Models

DiffWave: A Versatile Diffusion Model for Audio Synthesis

Adversarial score matching and improved sampling for image generation

WaveGrad: Estimating Gradients for Waveform Generation

Instance Selection for GANs

Generative Pretraining From Pixels

NVAE: A Deep Hierarchical Variational Autoencoder

Denoising Diffusion Probabilistic Models

Improved Techniques for Training Score-Based Generative Models

Language Models are Few-Shot Learners

Jukebox: A Generative Model for Music

Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One

Analyzing and Improving the Image Quality of StyleGAN

PyTorch: An Imperative Style, High-Performance Deep Learning Library

LOGAN: Latent Optimisation for Generative Adversarial Networks

Generative Modeling by Estimating Gradients of the Data Distribution

Large Scale Adversarial Representation Learning

Image Synthesis with a Single (Robust) Classifier

Generating Diverse High-Fidelity Images with VQ-VAE-2

Improved Precision and Recall Metric for Assessing Generative Models

High-Fidelity Image Generation With Fewer Labels

Hierarchical Autoregressive Image Models with Auxiliary Decoders

A Style-Based Generator Architecture for Generative Adversarial Networks

Large Scale GAN Training for High Fidelity Natural Image Synthesis

Group Normalization

Spectral Normalization for Generative Adversarial Networks

cGANs with Projection Discriminator

A Note on the Inception Score

Decoupled Weight Decay Regularization

Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Net

Neural Discrete Representation Learning

Detection of Online Fake News Using N-Gram Analysis and Machine Learning Techniques

Mixed Precision Training

FiLM: Visual Reasoning with a General Conditioning Layer

Modulating early visual processing by language

GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium

Attention is All you Need

StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks

RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation

A Learned Representation For Artistic Style

Neural Photo Editing with Introspective Adversarial Networks

WaveNet: A Generative Model for Raw Audio

Improved Techniques for Training GANs

A Theory of Generative ConvNet

Rethinking the Inception Architecture for Computer Vision

LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop

U-Net: Convolutional Networks for Biomedical Image Segmentation

Deep Unsupervised Learning using Nonequilibrium Thermodynamics

Adam: A Method for Stochastic Optimization

Deep Learning Face Attributes in the Wild

Conditional Generative Adversarial Nets

ImageNet Large Scale Visual Recognition Challenge

Intriguing properties of neural networks

Bayesian Learning via Stochastic Gradient Langevin Dynamics

Tooth and Claw

Training Products of Experts by Minimizing Contrastive Divergence

The Helmholtz Machine

Adverb

Protecting World Leaders Against Deep Fakes

NVIDIA

Implicit Generation and Generalization with Energy Based Models

Detecting opinion spams and fake news using text classification