Recycle-GAN: Unsupervised Video Retargeting (2018-08-15T00:00:00.000000Z)

TL;DR

This work introduces a data-driven approach for unsupervised video retargeting that translates content from one domain to another while preserving the style native to a domain, i.e., if contents of John Oliver's speech were to be transferred to Stephen Colbert, then the generated content/speech should be in Stephen Colbert's style.

Abstract

We introduce a data-driven approach for unsupervised video retargeting that translates content from one domain to another while preserving the style native to a domain, i.e., if contents of John Oliver's speech were to be transferred to Stephen Colbert, then the generated content/speech should be in Stephen Colbert's style. Our approach combines both spatial and temporal information along with adversarial losses for content translation and style preservation. In this work, we first study the advantages of using spatiotemporal constraints over spatial constraints for effective retargeting. We then demonstrate the proposed approach for the problems where information in both space and time matters such as face-to-face translation, flower-to-flower, wind and cloud synthesis, sunrise and sunset.

Authors

Deva Ramanan

21 papers

Yaser Sheikh

8 papers

Shugao Ma

2 papers

TL;DR

Abstract

Authors

References53 items

Unsupervised Video-to-Video Translation

Multimodal Unsupervised Image-to-Image Translation

Probabilistic Video Generation using Holistic Attribute Control

CyCADA: Cycle-Consistent Adversarial Domain Adaptation

UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Loss

Playing for Benchmarks

PixelNN: Example-based Image Synthesis

MoCoGAN: Decomposing Motion and Content for Video Generation

The Pose Knows: Video Forecasting by Generating Pose Futures

Unsupervised Learning of Depth and Ego-Motion from Video

Learning to Generate Long-term Future via Hierarchical Prediction

ActionVLAD: Learning Spatio-Temporal Aggregation for Action Classification

Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks

Learning to Discover Cross-Domain Relations with Generative Adversarial Networks

Learning from Simulated and Unsupervised Images through Adversarial Training

Stacked Generative Adversarial Networks

StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks

Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields

Image-to-Image Translation with Conditional Adversarial Networks

Temporal Generative Adversarial Nets with Singular Value Clipping

Dual Learning for Machine Translation

Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

Unsupervised Monocular Depth Estimation with Left-Right Consistency

Image Style Transfer Using Convolutional Neural Networks

Face2Face: Real-Time Face Capture and Reenactment of RGB Videos

An Uncertain Future: Forecasting from Static Images Using Variational Autoencoders

WarpNet: Weakly Supervised Matching for Single-View Reconstruction

Learning Dense Correspondence via 3D-Guided Cycle Consistency

The Cityscapes Dataset for Semantic Urban Scene Understanding

Marr Revisited: 2D-3D Alignment via Surface Normal Prediction

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

Real-time expression transfer for facial reenactment

Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks

FlowWeb: Joint image set alignment by weaving consistent, pixel-wise correspondences

Watch and learn: Semi-supervised learning of object detectors from videos

U-Net: Convolutional Networks for Biomedical Image Segmentation

Holistically-Nested Edge Detection

Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-scale Convolutional Architecture

Fully convolutional networks for semantic segmentation

Do Convnets Learn Correspondence?

ImageNet Large Scale Visual Recognition Challenge

3D object manipulation in a single photograph using stock 3D models

Displaced dynamic expression regression for real-time facial tracking and animation

Auto-Encoding Variational Bayes

Consistent Shape Maps via Semidefinite Programming

SIFT Flow: Dense Correspondence across Scenes and Its Applications

Beyond Categories: The Visual Memex Model for Reasoning About Object Relationships

Style translation for human motion

Image Analogies

Learning bilinear models for two-factor problems in vision

The Ecological Approach to Visual Perception

Style Machines Style Machines

Recycle-GAN

Field of Study

Venue Information

Name

Type

URL

Alternate Names