Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network

Published in

European Conference on Computer Vision(2018)

External Links:

Generate Graph

TL;DR

A straightforward method that simultaneously reconstructs the 3D facial structure and provides dense alignment and surpasses other state-of-the-art methods on both reconstruction and alignment tasks by a large margin.

Abstract

We propose a straightforward method that simultaneously reconstructs the 3D facial structure and provides dense alignment. To achieve this, we design a 2D representation called UV position map which records the 3D shape of a complete face in UV space, then train a simple Convolutional Neural Network to regress it from a single 2D image. We also integrate a weight mask into the loss function during training to improve the performance of the network. Our method does not rely on any prior face model, and can reconstruct full facial geometry along with semantic meaning. Meanwhile, our network is very light-weighted and spends only 9.8ms to process an image, which is extremely faster than previous works. Experiments on multiple challenging datasets show that our method surpasses other state-of-the-art methods on both reconstruction and alignment tasks by a large margin.

Authors

Yanfeng Wang

7 papers

Yao Feng

2 papers

Fan Wu

1 papers

References73 items

Side Information for Face Completion: A Robust PCA Approach

Multi-Attribute Robust Component Analysis for Facial UV Maps

UV-GAN: Adversarial Facial UV Map Completion for Pose-Invariant Face Recognition

Dense Face Alignment

Learning Dense Facial Correspondences in Unconstrained Images

Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network

Published in

European Conference on Computer Vision(2018)

External Links:

Generate Graph

TL;DR

Abstract

Authors

Yanfeng Wang

7 papers

Yao Feng

2 papers

Fan Wu

1 papers

References73 items

Side Information for Face Completion: A Robust PCA Approach

Multi-Attribute Robust Component Analysis for Facial UV Maps

UV-GAN: Adversarial Facial UV Map Completion for Pose-Invariant Face Recognition

Dense Face Alignment

Learning Dense Facial Correspondences in Unconstrained Images

Xiaohu Shao

1 papers

Pix2Face: Direct 3D Face Model Estimation

3D Morphable Models as Spatial Transformer Networks

Faster than Real-Time Facial Alignment: A 3D Spatial Transformer Network Approach in Unconstrained Poses

Fast 3D Reconstruction of Faces with Glasses

End-to-End 3D Face Reconstruction with Deep Neural Networks

MoFA: Model-Based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction

Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translation

Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression

How Far are We from Solving the 2D & 3D Face Alignment Problem? (and a Dataset of 230,000 3D Facial Landmarks)

SurfNet: Generating 3D Shape Surfaces Using Deep Residual Networks

Regressing Robust and Discriminative 3D Morphable Models with a Very Deep Neural Network

DenseReg: Fully Convolutional Dense Shape Regression In-the-Wild

A Point Set Generation Network for 3D Object Reconstruction from a Single Image

Learning Detailed Face Reconstruction from a Single Image

Joint Face Alignment and 3D Face Reconstruction with Application to Face Recognition

Joint Face Detection and Alignment with a Deformable Hough Transform Model

3D Face Alignment in the Wild: A Landmark-Free, Nose-Based Approach

The First 3D Face Alignment in the Wild (3DFAW) Challenge

3D Face Alignment Without Correspondences

Shape Augmented Regression for 3D Face Alignment

Fully Automated and Highly Accurate Dense Correspondence for Facial Surfaces

Face Reconstruction on Mobile Devices Using a Height Map Shape Model and Fast Regularization

Two-Stage Convolutional Part Heatmap Regression for the 1st 3D Face Alignment in the Wild (3DFAW) Challenge

Production-level facial performance capture using deep convolutional neural networks

Facial Performance Capture with Deep Neural Networks

3D Face Reconstruction by Learning from Synthetic Data

A Recurrent Encoder-Decoder Network for Sequential Face Alignment

Large-Pose Face Alignment via CNN-Based Dense 3D Model Fitting

Face2Face: Real-Time Face Capture and Reenactment of RGB Videos

Real-Time Facial Segmentation and Performance Capture from RGB Input

A Multiresolution 3D Morphable Face Model and Fitting Framework

Deep Residual Learning for Image Recognition

Offline Deformable Face Tracking in Arbitrary Videos

Unconstrained Facial Landmark Localization with Backbone-Branches Fully-Convolutional Networks

Pose-Invariant 3D Face Alignment

Global supervised descent method

High-fidelity Pose and Expression Normalization for face recognition in the wild

Dense 3D face alignment from 2D videos in real-time

Fitting 3D Morphable Face Models using local features

Eigenvector-based initial ranging process for OFDMA uplink systems

Optimal UV spaces for facial morphable model construction

Facial Landmark Detection by Deep Multi-task Learning

Displaced dynamic expression regression for real-time facial tracking and animation

Extensive Facial Landmark Localization with Coarse-to-Fine Convolutional Network Cascade

Optimization Problems for Fast AAM Fitting in-the-Wild

Viewing Real-World Faces in 3D

Deformable Spatial Pyramid Matching for Fast Dense Correspondences

Robust Discriminative Response Map Fitting with Constrained Local Models

Single view-based 3D face reconstruction robust to self-occlusion

Toward a Practical Face Recognition System: Robust Alignment and Illumination by Sparse Representation

Joint DOA and multi-pitch estimation based on subspace techniques

The florence 2D/3D hybrid face dataset

Annotated Facial Landmarks in the Wild: A large-scale, real-world database for facial landmark localization

Cascaded pose regression

A Nonlinear Discriminative Approach to AAM Fitting

A 3D facial expression database for facial behavior research

Estimating 3D shape and texture using pixel intensity, edges, specular highlights, texture constraints and a prior

Active Appearance Models Revisited

A Morphable Model For The Synthesis Of 3D Faces

Parametrization and smooth approximation of surface triangulations

Fast and Precise Face Alignment and 3D Shape Reconstruction from a Single 2D Image

Face alignment across large poses: A 3d solution. computer vision and pattern recognition

Multiple View Geometry in Computer Vision

Ieee Transactions on Pattern Analysis and Machine Intelligence 1 3d Face Reconstruction from a Single Image Using a Single Reference Face Shape

International Conference on Computer Vision and Pattern Recognition , New York , NY , June 17-22 , 2006 1 3 D Alignment of Face in a Single Image Lie Gu and Takeo Kanade Computer Science

This Paper Is Included in the Proceedings of the 12th Usenix Symposium on Operating Systems Design and Implementation (osdi '16). Tensorflow: a System for Large-scale Machine Learning Tensorflow: a System for Large-scale Machine Learning

– For the ﬁrst time, we solve the problems of face alignment and 3D face reconstruction together in an end-to-end fashion without the restriction of low-dimensional solution space

Field of Study

Computer Science

Venue Information

Name

European Conference on Computer Vision

Type

conference

URL

https://link.springer.com/conference/eccv

Alternate Names

ECCV
Eur Conf Comput Vis

TL;DR

Abstract

Authors

References73 items

Side Information for Face Completion: A Robust PCA Approach

Multi-Attribute Robust Component Analysis for Facial UV Maps

UV-GAN: Adversarial Facial UV Map Completion for Pose-Invariant Face Recognition

Dense Face Alignment

Learning Dense Facial Correspondences in Unconstrained Images

TL;DR

Abstract

Authors

References73 items

Side Information for Face Completion: A Robust PCA Approach

Multi-Attribute Robust Component Analysis for Facial UV Maps

UV-GAN: Adversarial Facial UV Map Completion for Pose-Invariant Face Recognition

Dense Face Alignment

Learning Dense Facial Correspondences in Unconstrained Images

Pix2Face: Direct 3D Face Model Estimation

3D Morphable Models as Spatial Transformer Networks

Faster than Real-Time Facial Alignment: A 3D Spatial Transformer Network Approach in Unconstrained Poses

Fast 3D Reconstruction of Faces with Glasses

End-to-End 3D Face Reconstruction with Deep Neural Networks

MoFA: Model-Based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction

Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translation

Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression

How Far are We from Solving the 2D & 3D Face Alignment Problem? (and a Dataset of 230,000 3D Facial Landmarks)

SurfNet: Generating 3D Shape Surfaces Using Deep Residual Networks

Regressing Robust and Discriminative 3D Morphable Models with a Very Deep Neural Network

DenseReg: Fully Convolutional Dense Shape Regression In-the-Wild

A Point Set Generation Network for 3D Object Reconstruction from a Single Image

Learning Detailed Face Reconstruction from a Single Image

Joint Face Alignment and 3D Face Reconstruction with Application to Face Recognition

Joint Face Detection and Alignment with a Deformable Hough Transform Model

3D Face Alignment in the Wild: A Landmark-Free, Nose-Based Approach

The First 3D Face Alignment in the Wild (3DFAW) Challenge

3D Face Alignment Without Correspondences

Shape Augmented Regression for 3D Face Alignment

Fully Automated and Highly Accurate Dense Correspondence for Facial Surfaces

Face Reconstruction on Mobile Devices Using a Height Map Shape Model and Fast Regularization

Two-Stage Convolutional Part Heatmap Regression for the 1st 3D Face Alignment in the Wild (3DFAW) Challenge

Production-level facial performance capture using deep convolutional neural networks

Facial Performance Capture with Deep Neural Networks

3D Face Reconstruction by Learning from Synthetic Data

A Recurrent Encoder-Decoder Network for Sequential Face Alignment

Large-Pose Face Alignment via CNN-Based Dense 3D Model Fitting

Face2Face: Real-Time Face Capture and Reenactment of RGB Videos

Real-Time Facial Segmentation and Performance Capture from RGB Input

A Multiresolution 3D Morphable Face Model and Fitting Framework

Deep Residual Learning for Image Recognition

Offline Deformable Face Tracking in Arbitrary Videos

Unconstrained Facial Landmark Localization with Backbone-Branches Fully-Convolutional Networks

Pose-Invariant 3D Face Alignment

Global supervised descent method

High-fidelity Pose and Expression Normalization for face recognition in the wild

Dense 3D face alignment from 2D videos in real-time

Fitting 3D Morphable Face Models using local features

Eigenvector-based initial ranging process for OFDMA uplink systems

Optimal UV spaces for facial morphable model construction

Facial Landmark Detection by Deep Multi-task Learning

Displaced dynamic expression regression for real-time facial tracking and animation

Extensive Facial Landmark Localization with Coarse-to-Fine Convolutional Network Cascade

Optimization Problems for Fast AAM Fitting in-the-Wild

Viewing Real-World Faces in 3D

Deformable Spatial Pyramid Matching for Fast Dense Correspondences

Robust Discriminative Response Map Fitting with Constrained Local Models

Single view-based 3D face reconstruction robust to self-occlusion

Toward a Practical Face Recognition System: Robust Alignment and Illumination by Sparse Representation

Joint DOA and multi-pitch estimation based on subspace techniques

The florence 2D/3D hybrid face dataset

Annotated Facial Landmarks in the Wild: A large-scale, real-world database for facial landmark localization

Cascaded pose regression

A Nonlinear Discriminative Approach to AAM Fitting

A 3D facial expression database for facial behavior research

Estimating 3D shape and texture using pixel intensity, edges, specular highlights, texture constraints and a prior

Active Appearance Models Revisited

Geometry images

A Morphable Model For The Synthesis Of 3D Faces

Parametrization and smooth approximation of surface triangulations

Fast and Precise Face Alignment and 3D Shape Reconstruction from a Single 2D Image