Multimodal Image Outpainting with Regularized Normalized Diversification (2019-10-25T00:00:00.000000Z)

TL;DR

This paper proposes a new regularization method to encourage diverse sampling in conditional synthesis and proposes a feature pyramid discriminator to improve the image quality.

Abstract

In this paper, we study the problem of generating a set of realistic and diverse backgrounds when given only a small foreground region. We refer to this task as image outpainting. The technical challenge of this task is to synthesize not only plausible but also diverse image outputs. Traditional generative adversarial networks suffer from mode collapse. While recent approaches [32], [28] propose to maximize or preserve the pairwise distance between generated samples with respect to their latent distance, they do not explicitly prevent the diverse samples of different conditional inputs from collapsing. Therefore, we propose a new regularization method to encourage diverse sampling in conditional synthesis. In addition, we propose a feature pyramid discriminator to improve the image quality. Our experimental results show that our model can produce more diverse images without sacrificing visual quality compared to state-of-the-arts approaches in both the CelebA face dataset [29] and the Cityscape scene dataset [2]. Code is available at: https://github.com/owenzlz/DiverseOutpaint

Authors

Lingzhi Zhang

3 papers

Jianbo Shi

5 papers

Jiancong Wang

1 papers

TL;DR

Abstract

Authors

References53 items

Generative Adversarial Networks

Normalized Diversification

Mode Seeking Generative Adversarial Networks for Diverse Image Synthesis

Putting Humans in a Scene: Learning Affordance in 3D Indoor Environments

Object-Driven Text-To-Image Synthesis via Adversarial Training

EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning

Context-aware Synthesis and Placement of Object Instances

ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness

ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks

BourGAN: Generative Networks with Metric Embeddings

Image Inpainting for Irregular Holes Using Partial Convolutions

Pedestrian-Synthesis-GAN: Generating Pedestrian Data in Real Scene and Beyond

Image Generation from Scene Graphs

ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositing

Spectral Normalization for Generative Adversarial Networks

Generative Image Inpainting with Contextual Attention

The Unreasonable Effectiveness of Deep Features as a Perceptual Metric

Im2Flow: Motion Hallucination from Static Images for Action Recognition

Super-FAN: Integrated Facial Landmark Localization and Super-Resolution of Real-World Low Resolution Faces in Arbitrary Poses with GANs

Toward Multimodal Image-to-Image Translation

Globally and locally consistent image completion

Binge Watching: Scaling Affordance Learning from Sitcoms

Hallucinating Very Low-Resolution Unaligned and Noisy Face Images by Transformative Discriminative Autoencoders

GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium

Universal Style Transfer via Feature Transforms

Deep Photo Style Transfer

Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization

Mask R-CNN

Face Hallucination with Tiny Unaligned Images by Transformative Discriminative Neural Networks

Pyramid Scene Parsing Network

Image-to-Image Translation with Conditional Adversarial Networks

Ultra-Resolving Face Images by Discriminative Generative Networks

A Multi-scale CNN for Affordance Segmentation in RGB Images

Semantic Image Inpainting with Deep Generative Models

Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks

Image Style Transfer Using Convolutional Neural Networks

An Uncertain Future: Forecasting from Static Images Using Variational Autoencoders

Adversarial Feature Learning

Generative Adversarial Text to Image Synthesis

Context Encoders: Feature Learning by Inpainting

The Cityscapes Dataset for Semantic Urban Scene Understanding

Perceptual Losses for Real-Time Style Transfer and Super-Resolution

Autoencoding beyond pixels using a learned similarity metric

Accurate Image Super-Resolution Using Very Deep Convolutional Networks

Image Super-Resolution Using Deep Convolutional Networks

Deep Convolutional Neural Network for Image Deconvolution

Deep Learning Face Attributes in the Wild

Very Deep Convolutional Networks for Large-Scale Image Recognition

Mask-Specific Inpainting with Deep Neural Networks

Auto-Encoding Variational Bayes

People Watching: Human Actions as a Cue for Single View Geometry

From 3D scene geometry to human workspace

Visualizing Data using t-SNE

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names