Interpreting the Latent Space of GANs for Semantic Face Editing (2019-07-25T00:00:00.000000Z)

TL;DR

This work proposes a novel framework, called InterFaceGAN, for semantic face editing by interpreting the latent semantics learned by GANs, and finds that the latent code of well-trained generative models actually learns a disentangled representation after linear transformations.

Abstract

Despite the recent advance of Generative Adversarial Networks (GANs) in high-fidelity image synthesis, there lacks enough understanding of how GANs are able to map a latent code sampled from a random distribution to a photo-realistic image. Previous work assumes the latent space learned by GANs follows a distributed representation but observes the vector arithmetic phenomenon. In this work, we propose a novel framework, called InterFaceGAN, for semantic face editing by interpreting the latent semantics learned by GANs. In this framework, we conduct a detailed study on how different semantics are encoded in the latent space of GANs for face synthesis. We find that the latent code of well-trained generative models actually learns a disentangled representation after linear transformations. We explore the disentanglement between various semantics and manage to decouple some entangled semantics with subspace projection, leading to more precise control of facial attributes. Besides manipulating gender, age, expression, and the presence of eyeglasses, we can even vary the face pose as well as fix the artifacts accidentally generated by GAN models. The proposed method is further applied to achieve real image manipulation when combined with GAN inversion methods or some encoder-involved models. Extensive results suggest that learning to synthesize faces spontaneously brings a disentangled and controllable facial attribute representation.

Authors

Xiaoou Tang

26 papers

Bolei Zhou

20 papers

Jinjin Gu

8 papers

TL;DR

Abstract

Authors

References47 items

Foundations of Data Science

Image Processing Using Multi-Code GAN Prior

Seeing What a GAN Cannot Generate

Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesis

LIA: Latently Invertible Autoencoder with Adversarial Learning

On the "steerability" of generative adversarial networks

GANalyze: Toward Visual Definitions of Cognitive Image Properties

Disentangled Inference for GANs With Latently Invertible Autoencoder

Visualizing and Understanding Generative Adversarial Networks (Extended Abstract)

A Style-Based Generator Architecture for Generative Adversarial Networks

FaceFeat-GAN: a Two-Stage Approach for Identity-Preserving Face Synthesis

Large Scale GAN Training for High Fidelity Natural Image Synthesis

FaceID-GAN: Learning a Symmetry Three-Player GAN for Identity-Preserving Face Synthesis

Self-Attention Generative Adversarial Networks

Latent Space Non-Linear Statistics

Towards Open-Set Identity Preserving Face Synthesis

ELEGANT: Exchanging Latent Encodings with GAN for Transferring Multiple Face Attributes

Spectral Normalization for Generative Adversarial Networks

Feature-Based Metrics for Exploring the Latent Space of Generative Models

The Riemannian Geometry of Deep Generative Models

Metrics for Deep Generative Models

Latent Space Oddity: on the Curvature of Deep Generative Models

Progressive Growing of GANs for Improved Quality, Stability, and Variation

Optimizing the Latent Space of Generative Networks

Wasserstein Generative Adversarial Networks

Disentangled Representation Learning GAN for Pose-Invariant Face Recognition

Fader Networks: Manipulating Images by Sliding Attributes

Semantically Decomposing the Latent Spaces of Generative Adversarial Networks

Towards Large-Pose Face Frontalization in the Wild

Improved Training of Wasserstein GANs

BEGAN: Boundary Equilibrium Generative Adversarial Networks

Wasserstein GAN

StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks

Invertible Conditional GANs for image editing

Deep Feature Interpolation for Image Content Changes

Conditional Image Synthesis with Auxiliary Classifier GANs

Generative Visual Manipulation on the Natural Image Manifold

InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets

Adversarially Learned Inference

Adversarial Feature Learning

Deep Residual Learning for Image Recognition

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

Deep Learning Face Attributes in the Wild

Conditional Generative Adversarial Nets

Invertibility of Convolutional Generative Networks from Partial Measurements

GENERATIVE ADVERSARIAL NETS

Influence of cultivation temperature on the ligninolytic activity of selected fungal strains

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names