ReenactGAN: Learning to Reenact Faces via Boundary Transfer (2018-07-29T00:00:00.000000Z)

TL;DR

The proposed method, known as ReenactGAN, is capable of transferring facial movements and expressions from an arbitrary person's monocular video input to a target person’s video, and can perform photo-realistic face reenactment.

Abstract

We present a novel learning-based framework for face reenactment. The proposed method, known as ReenactGAN, is capable of transferring facial movements and expressions from an arbitrary person’s monocular video input to a target person’s video. Instead of performing a direct transfer in the pixel space, which could result in structural artifacts, we first map the source face onto a boundary latent space. A transformer is subsequently used to adapt the source face’s boundary to the target’s boundary. Finally, a target-specific decoder is used to generate the reenacted target face. Thanks to the effective and reliable boundary-based transfer, our method can perform photo-realistic face reenactment. In addition, ReenactGAN is appealing in that the whole reenactment process is purely feed-forward, and thus the reenactment process can run in real-time (30 FPS on one GTX 1080 GPU). Dataset and model are publicly available on our project page (Project Page: https://wywu.github.io/projects/ReenactGAN/ReenactGAN.html).

Authors

Chen Change Loy

72 papers

Wayne Wu

14 papers

C. Qian

8 papers

TL;DR

Abstract

Authors

References62 items

Deep video portraits

Look at Boundary: A Boundary-Aware Face Alignment Algorithm

Hallucinated-IQA: No-Reference Image Quality Assessment via Adversarial Learning

Style Aggregated Network for Facial Landmark Detection

Disentangling 3D Pose in a Dendritic CNN for Unconstrained 2D Face Alignment

DensePose: Dense Human Pose Estimation in the Wild

CycleGAN Face-off

GANerated Hands for Real-Time 3D Hand Tracking from Monocular RGB

High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs

Toward Multimodal Image-to-Image Translation

Face Transfer with Generative Adversarial Network

Be Your Own Prada: Fashion Synthesis with Structural Coherence

Deep Facial Action Unit Recognition from Partially Labeled Data

Joint Multi-View Face Alignment in the Wild

Quantifying Facial Age by Posterior of Age Comparisons

Leveraging Intra and Inter-Dataset Variations for Robust Face Alignment

Deep Structured Learning for Facial Action Unit Intensity Estimation

DeepCoder: Semi-Parametric Variational Autoencoders for Automatic Facial Action Coding

Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks

EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis

StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks

DenseReg: Fully Convolutional Dense Shape Regression In-the-Wild

Image-to-Image Translation with Conditional Adversarial Networks

Deep Feature Interpolation for Image Content Changes

Model-based teeth reconstruction

FaceVR: Real-Time Facial Reenactment and Eye Gaze Control in Virtual Reality

Robust Facial Landmark Detection via Recurrent Attentive-Refinement Networks

Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

Realtime 3D eye gaze animation using a single RGB camera

Face2Face: Real-Time Face Capture and Reenactment of RGB Videos

Unconstrained Face Alignment via Cascaded Compositional Learning

Mnemonic Descent Method: A Recurrent Process Applied for End-to-End Face Alignment

Deep Region and Multi-label Learning for Facial Action Unit Detection

Copula Ordinal Regression for Joint Estimation of Facial Action Unit Intensity

Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks

Stacked Hourglass Networks for Human Pose Estimation

What Makes Tom Hanks Look Like Tom Hanks

Real-time expression transfer for facial reenactment

Face alignment by coarse-to-fine shape searching

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

VDub: Modifying Face Video of Actors for Plausible Visual Alignment to a Dubbed Audio Track

Very Deep Convolutional Networks for Large-Scale Image Recognition

Automatic Face Reenactment

DeepFace: Closing the Gap to Human-Level Performance in Face Verification

3D shape regression for real-time facial animation

DISFA: A Spontaneous Facial Action Intensity Database

Interactive Facial Feature Localization

A data-driven approach for facial expression synthesis in video

Video face replacement

3D-model-based face replacement in video

A framework for automated measurement of the intensity of non-posed Facial Action Units

Facial Action Unit Recognition by Exploiting Their Dynamic and Semantic Relationships

TrueSkillTM: A Bayesian Skill Rating System

Automatic Recognition of Facial Actions in Spontaneous Expressions

Multimodal affect recognition in learning environments

Face transfer with multilinear models

Machine learning methods for fully automatic recognition of facial expressions and facial actions

Image quality assessment: from error visibility to structural similarity

Self-supervised Deep Multiple Choice Learning Network for Blind Image Quality Assessment

Supplementary materials for: Plug & Play Generative Networks: Conditional Iterative Generation of Images in Latent Space

Facial Action Coding System (FACS): Manual

Ieee Transactions on Pattern Analysis and Machine Intelligence High-speed Tracking with Kernelized Correlation Filters

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names