StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks

Published in

IEEE International Conference on Computer Vision(2016)

External Links:

Generate Graph DownloadPDF

TL;DR

This paper proposes Stacked Generative Adversarial Networks (StackGAN) to generate 256 photo-realistic images conditioned on text descriptions and introduces a novel Conditioning Augmentation technique that encourages smoothness in the latent conditioning manifold.

Abstract

Synthesizing high-quality images from text descriptions is a challenging problem in computer vision and has many practical applications. Samples generated by existing textto- image approaches can roughly reflect the meaning of the given descriptions, but they fail to contain necessary details and vivid object parts. In this paper, we propose Stacked Generative Adversarial Networks (StackGAN) to generate 256.256 photo-realistic images conditioned on text descriptions. We decompose the hard problem into more manageable sub-problems through a sketch-refinement process. The Stage-I GAN sketches the primitive shape and colors of the object based on the given text description, yielding Stage-I low-resolution images. The Stage-II GAN takes Stage-I results and text descriptions as inputs, and generates high-resolution images with photo-realistic details. It is able to rectify defects in Stage-I results and add compelling details with the refinement process. To improve the diversity of the synthesized images and stabilize the training of the conditional-GAN, we introduce a novel Conditioning Augmentation technique that encourages smoothness in the latent conditioning manifold. Extensive experiments and comparisons with state-of-the-arts on benchmark datasets demonstrate that the proposed method achieves significant improvements on generating photo-realistic images conditioned on text descriptions.

Authors

Xiaogang Wang

21 papers

Xiaolei Huang

2 papers

Tao Xu

1 papers

References46 items

Generating Interpretable Images with Controllable Structure

Towards Principled Methods for Training Generative Adversarial Networks

Stacked Generative Adversarial Networks

Image-to-Image Translation with Conditional Adversarial Networks

Mode Regularized Generative Adversarial Networks

StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks

Published in

IEEE International Conference on Computer Vision(2016)

External Links:

Generate Graph DownloadPDF

TL;DR

Abstract

Authors

Xiaogang Wang

21 papers

Xiaolei Huang

2 papers

Tao Xu

1 papers

References46 items

Generating Interpretable Images with Controllable Structure

Towards Principled Methods for Training Generative Adversarial Networks

Stacked Generative Adversarial Networks

Image-to-Image Translation with Conditional Adversarial Networks

Mode Regularized Generative Adversarial Networks

Hongsheng Li

26 papers

Shaoting Zhang

1 papers

Dimitris N. Metaxas

9 papers

Unrolled Generative Adversarial Networks

Unsupervised Cross-Domain Image Generation

Conditional Image Synthesis with Auxiliary Classifier GANs

Amortised MAP Inference for Image Super-resolution

Learning What and Where to Draw

Neural Photo Editing with Introspective Adversarial Networks

Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

Generative Visual Manipulation on the Natural Image Manifold

Energy-based Generative Adversarial Network

Tutorial on Variational Autoencoders

Conditional Image Generation with PixelCNN Decoders

InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets

Improved Techniques for Training GANs

Generative Adversarial Text to Image Synthesis

Learning Deep Representations of Fine-Grained Visual Descriptions

Perceptual Losses for Real-Time Style Transfer and Super-Resolution

Generative Image Modeling Using Style and Structure Adversarial Networks

Pixel Recurrent Neural Networks

Autoencoding beyond pixels using a learned similarity metric

Deep Residual Learning for Image Recognition

Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesis

Deep Visual Analogy-Making

Attribute2Image: Conditional Image Generation from Visual Attributes

Rethinking the Inception Architecture for Computer Vision

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

Generating Images from Captions with Attention

Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks

Empirical Evaluation of Rectified Activations in Convolutional Network

DRAW: A Recurrent Neural Network For Image Generation

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Learning to generate chairs with convolutional neural networks

Conditional Generative Adversarial Nets

Microsoft COCO: Common Objects in Context

Stochastic Backpropagation and Approximate Inference in Deep Generative Models

Auto-Encoding Variational Bayes

The Caltech-UCSD Birds-200-2011 Dataset

Automated Flower Classification over a Large Number of Classes

GENERATIVE ADVERSARIAL NETS

Supplementary materials for: Plug & Play Generative Networks: Conditional Iterative Generation of Images in Latent Space

Conditional generative adversarial nets for convolutional face generation

Rectifier Nonlinearities Improve Neural Network Acoustic Models

Field of Study

Computer ScienceMathematics

Journal Information

Name

2017 IEEE International Conference on Computer Vision (ICCV)

Page

5908-5916

Venue Information

Name

IEEE International Conference on Computer Vision

Type

conference

URL

https://ieeexplore.ieee.org/xpl/conhome/1000149/all-proceedings

Alternate Names

ICCV
IEEE Int Conf Comput Vis
ICCV Workshops
ICCV Work