Closed-Form Factorization of Latent Semantics in GANs (2020-07-13T00:00:00.000000Z)

TL;DR

This work examines the internal representation learned by GANs to reveal the underlying variation factors in an unsupervised manner and proposes a closedform factorization algorithm for latent semantic discovery by directly decomposing the pre-trained weights.

Abstract

A rich set of interpretable dimensions has been shown to emerge in the latent space of the Generative Adversarial Networks (GANs) trained for synthesizing images. In order to identify such latent dimensions for image editing, previous methods typically annotate a collection of synthesized samples and train linear classifiers in the latent space. However, they require a clear definition of the target attribute as well as the corresponding manual annotations, limiting their applications in practice. In this work, we examine the internal representation learned by GANs to reveal the underlying variation factors in an unsupervised manner. In particular, we take a closer look into the generation mechanism of GANs and further propose a closedform factorization algorithm for latent semantic discovery by directly decomposing the pre-trained weights. With a lightning-fast implementation, our approach is capable of not only finding semantically meaningful dimensions comparably to the state-of-the-art supervised methods, but also resulting in far more versatile concepts across multiple GAN models trained on a wide range of datasets.1

Authors

Bolei Zhou

20 papers

Yujun Shen

8 papers

TL;DR

Abstract

Authors

References30 items

InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs

GANSpace: Discovering Interpretable GAN Controls

In-Domain GAN Inversion for Real Image Editing

Unsupervised Discovery of Interpretable Directions in the GAN Latent Space

Controlling generative models with continuous factors of variations

Image Processing Using Multi-Code GAN Prior

Analyzing and Improving the Image Quality of StyleGAN

Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesis

Interpreting the Latent Space of GANs for Semantic Face Editing

On the "steerability" of generative adversarial networks

GANalyze: Toward Visual Definitions of Cognitive Image Properties

A Style-Based Generator Architecture for Generative Adversarial Networks

Large Scale GAN Training for High Fidelity Natural Image Synthesis

GAN Dissection: Visualizing and Understanding Generative Adversarial Networks

Progressive Growing of GANs for Improved Quality, Stability, and Variation

Wasserstein Generative Adversarial Networks

GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium

Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization

beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework

InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets

Deep Residual Learning for Image Recognition

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop

Deep Learning Face Attributes in the Wild

Streetscore -- Predicting the Perceived Safety of One Million Streetscapes

ImageNet: A large-scale hierarchical image database

Danbooru2019: A large-scale crowdsourced and tagged anime

GENERATIVE ADVERSARIAL NETS

Progressive training of gans with mutual information penalty

Towards the High-quality Anime Characters Generation with Generative Adversarial Networks

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names