Analyzing and Improving the Image Quality of StyleGAN (2019-12-03T00:00:00.000000Z)

TL;DR

This work redesigns the generator normalization, revisit progressive growing, and regularize the generator to encourage good conditioning in the mapping from latent codes to images, and thereby redefines the state of the art in unconditional image modeling.

Abstract

The style-based GAN architecture (StyleGAN) yields state-of-the-art results in data-driven unconditional generative image modeling. We expose and analyze several of its characteristic artifacts, and propose changes in both model architecture and training methods to address them. In particular, we redesign the generator normalization, revisit progressive growing, and regularize the generator to encourage good conditioning in the mapping from latent codes to images. In addition to improving image quality, this path length regularizer yields the additional benefit that the generator becomes significantly easier to invert. This makes it possible to reliably attribute a generated image to a particular network. We furthermore visualize how well the generator utilizes its output resolution, and identify a capacity problem, motivating us to train larger models for additional quality improvements. Overall, our improved model redefines the state of the art in unconditional image modeling, both in terms of existing distribution quality metrics as well as perceived image quality.

Authors

S. Laine

8 papers

Timo Aila

8 papers

Tero Karras

5 papers

TL;DR

Abstract

Authors

References52 items

CNN-Generated Images Are Surprisingly Easy to Spot… for Now

FakeSpotter: A Simple Baseline for Spotting AI-Synthesized Fake Faces

Interpreting the Latent Space of GANs for Semantic Face Editing

Detecting and Simulating Artifacts in GAN Fake Images

Style Generator Inversion for Image Enhancement and Animation

Source Generator Attribution via Inversion

Making Convolutional Networks Shift-Invariant Again

Improved Precision and Recall Metric for Assessing Generative Models

Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?

MSG-GAN: Multi-Scale Gradients for Generative Adversarial Networks

MSG-GAN: Multi-Scale Gradient GAN for Stable Image Synthesis

A Style-Based Generator Architecture for Generative Adversarial Networks

Attributing Fake Images to GANs: Analyzing Fingerprints in Generated Images

Can Forensic Detectors Identify GAN Generated Images?

Large Scale GAN Training for High Fidelity Natural Image Synthesis

ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness

Feature-wise transformations

Assessing Generative Models via Precision and Recall

Self-Attention Generative Adversarial Networks

Is Generator Conditioning Causally Related to GAN Performance?

Spectral Normalization for Generative Adversarial Networks

Which Training Methods for GANs do actually Converge?

The Unreasonable Effectiveness of Deep Features as a Perceptual Metric

Progressive Growing of GANs for Improved Quality, Stability, and Variation

StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks

GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium

Exploring the structure of a real-time, arbitrary neural artistic stylization network

On the Effects of Batch and Weight Normalization in Generative Adversarial Networks

Improved Training of Wasserstein GANs

Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization

StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks

A Learned Representation For Artistic Style

Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks

Deep Residual Learning for Image Recognition

Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks

LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop

U-Net: Convolutional Networks for Biomedical Image Segmentation

RMSProp and equilibrated adaptive learning rates for non-convex optimization

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

Adam: A Method for Stochastic Optimization

Very Deep Convolutional Networks for Large-Scale Image Recognition

ImageNet Large Scale Visual Recognition Challenge

ImageNet classification with deep convolutional neural networks

Understanding the difficulty of training deep feedforward neural networks

Pattern Recognition and Machine Learning

The importance of shape in early lexical learning

Pyramidal parametrics

Power measurement tutorial for the Green500 list. https://www.top500.org/green500/resources/tutorials

Which face is real? http://www.whichfaceisreal.com/learn.html

StyleGAN – Encoder for official TensorFlow implementation

Matrix Computations

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names