Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks (2017-03-30T00:00:00.000000Z)

TL;DR

This work presents an approach for learning to translate an image from a source domain X to a target domain Y in the absence of paired examples, and introduces a cycle consistency loss to push F(G(X)) ≈ X (and vice versa).

Abstract

Image-to-image translation is a class of vision and graphics problems where the goal is to learn the mapping between an input image and an output image using a training set of aligned image pairs. However, for many tasks, paired training data will not be available. We present an approach for learning to translate an image from a source domain X to a target domain Y in the absence of paired examples. Our goal is to learn a mapping G : X → Y such that the distribution of images from G(X) is indistinguishable from the distribution Y using an adversarial loss. Because this mapping is highly under-constrained, we couple it with an inverse mapping F : Y → X and introduce a cycle consistency loss to push F(G(X)) ≈ X (and vice versa). Qualitative results are presented on several tasks where paired training data does not exist, including collection style transfer, object transfiguration, season transfer, photo enhancement, etc. Quantitative comparisons against several prior methods demonstrate the superiority of our approach.

Authors

Phillip Isola

11 papers

Alexei A. Efros

23 papers

Jun-Yan Zhu

10 papers

TL;DR

Abstract

Authors

References68 items

DualGAN: Unsupervised Dual Learning for Image-to-Image Translation

Unsupervised Image-to-Image Translation Networks

NIPS 2016 Tutorial: Generative Adversarial Networks

Learning from Simulated and Unsupervised Images through Adversarial Training

Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks

Scribbler: Controlling Deep Image Synthesis with Sketch and Color

Learning to Generate Images of Outdoor Scenes from Attributes and Semantic Layouts

Image-to-Image Translation with Conditional Adversarial Networks

Least Squares Generative Adversarial Networks

Multi-class Generative Adversarial Networks with the L2 Loss Function

Disentangling factors of variation in deep representation using adversarial training

Unsupervised Cross-Domain Image Generation

Dual Learning for Machine Translation

Cross-Modal Scene Networks

Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

Unsupervised Monocular Depth Estimation with Left-Right Consistency

Generative Visual Manipulation on the Natural Image Manifold

Energy-based Generative Adversarial Network

Generating Videos with Scene Dynamics

Instance Normalization: The Missing Ingredient for Fast Stylization

Image Style Transfer Using Convolutional Neural Networks

Coupled Generative Adversarial Networks

Preserving Color in Neural Artistic Style Transfer

Improved Techniques for Training GANs

Adversarially Learned Inference

Adversarial Feature Learning

Generative Adversarial Text to Image Synthesis

Context Encoders: Feature Learning by Inpainting

Learning Dense Correspondence via 3D-Guided Cycle Consistency

Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks

The Cityscapes Dataset for Semantic Urban Scene Understanding

Colorful Image Colorization

Perceptual Losses for Real-Time Style Transfer and Super-Resolution

Generative Image Modeling Using Style and Structure Adversarial Networks

Texture Networks: Feed-forward Synthesis of Textures and Stylized Images

Generating Images with Perceptual Similarity Metrics based on Deep Networks

Deep Residual Learning for Image Recognition

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

Adversarial Autoencoders

Deep multi-scale video prediction beyond mean square error

Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks

Modeling object appearance using Context-Conditioned Component Analysis

FlowWeb: Joint image set alignment by weaving consistent, pixel-wise correspondences

Holistically-Nested Edge Detection

Adam: A Method for Stochastic Optimization

Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-scale Convolutional Architecture

Fully convolutional networks for semantic segmentation

Very Deep Convolutional Networks for Large-Scale Image Recognition

ImageNet Large Scale Visual Recognition Challenge

Transient attributes for high-level understanding and editing of outdoor scenes

Fine-Grained Visual Comparisons with Local Learning

Auto-Encoding Variational Bayes

Image Co-segmentation via Consistent Functional Maps

Data-driven hallucination of different times of day from a single outdoor photo

Spatial Pattern Templates for Recognition of Objects with Regular Structure

Consistent Shape Maps via Semidefinite Programming

Dense Point Trajectories by GPU-Accelerated Large Displacement Optical Flow

Forward-Backward Error: Automatic Detection of Tracking Failures

Disambiguating visual relations using loop constraints

ImageNet: A large-scale hierarchical image database

Unsupervised image translation

Image Analogies

Texture synthesis by non-parametric sampling

Back-Translation for Cross-Cultural Research

GENERATIVE ADVERSARIAL NETS

Supporting Online Material for Reducing the Dimensionality of Data with Neural Networks

The jumping frog : in English, then in French, then clawed back into a civilized language once more by patient, unremunerated toil

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL