Fast Bi-layer Neural Synthesis of One-Shot Realistic Head Avatars (2020-08-23T00:00:00.000000Z)

TL;DR

A neural rendering-based system that creates head avatars from a single photograph by decomposing it into two layers that is compared to analogous state-of-the-art systems in terms of visual quality and speed.

Abstract

We propose a neural rendering-based system that creates head avatars from a single photograph. Our approach models a person's appearance by decomposing it into two layers. The first layer is a pose-dependent coarse image that is synthesized by a small neural network. The second layer is defined by a pose-independent texture image that contains high-frequency details. The texture image is generated offline, warped and added to the coarse image to ensure a high effective resolution of synthesized head views. We compare our system to analogous state-of-the-art systems in terms of visual quality and speed. The experiments show significant inference speedup over previous neural head avatar models for a given visual quality. We also report on a real-time smartphone-based implementation of our system.

Authors

V. Lempitsky

14 papers

Egor Zakharov

2 papers

Aliaksandra Shysheya

2 papers

TL;DR

Abstract

Authors

References40 items

First Order Motion Model for Image Animation

MarioNETte: Few-shot Face Reenactment Preserving Identity of Unseen Targets

Few-shot Video-to-Video Synthesis

GRDN:Grouped Residual Dense Network for Real Image Denoising and GAN-Based Real-World Noise Modeling

Textured Neural Avatars

Few-Shot Adversarial Learning of Realistic Neural Talking Head Models

Graphonomy: Universal Human Parsing via Graph Transfer Learning

ICface: Interpretable and Controllable Face Reenactment Using GANs

High Fidelity Face Manipulation with Extreme Pose and Expression

Large Scale GAN Training for High Fidelity Natural Image Synthesis

Dense Pose Transfer

Video-to-Video Synthesis

Deep appearance models for face rendering

X2Face: A network for controlling face generation by using images, audio, and pose codes

The relativistic discriminator: a key element missing from standard GAN

VoxCeleb2: Deep Speaker Recognition

Deep video portraits

ArcFace: Additive Angular Margin Loss for Deep Face Recognition

The Unreasonable Effectiveness of Deep Features as a Perceptual Metric

High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs

Avatar digitization from a single image for real-time rendering

Synthesizing Obama

How Far are We from Solving the 2D & 3D Face Alignment Problem? (and a Dataset of 230,000 3D Facial Landmarks)

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

Image-to-Image Translation with Conditional Adversarial Networks

DeepWarp: Photorealistic Image Resynthesis for Gaze Manipulation

Learning to learn by gradient descent by gradient descent

View Synthesis by Appearance Flow

Perceptual Losses for Real-Time Style Transfer and Super-Resolution

Identity Mappings in Deep Residual Networks

Spatial Transformer Networks

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Adam: A Method for Stochastic Optimization

Learning to generate chairs with convolutional neural networks

Conditional Generative Adversarial Nets

Very Deep Convolutional Networks for Large-Scale Image Recognition

Generative Adversarial Nets

The Digital Emily Project: Achieving a Photorealistic Digital Actor

Deep Face Recognition

PyTorch homepage

Field of Study

Venue Information

Name

Type

URL

Alternate Names