Perspective Reconstruction of Human Faces by Joint Mesh and Landmark Regression (2022-08-15T00:00:00.000000Z)

TL;DR

This paper proposes to simultaneously reconstruct3D face mesh in the world space and predict 2D face landmarks on the image plane to address the problem of perspective 3D face reconstruction.

Abstract

Even though 3D face reconstruction has achieved impressive progress, most orthogonal projection-based face reconstruction methods can not achieve accurate and consistent reconstruction results when the face is very close to the camera due to the distortion under the perspective projection. In this paper, we propose to simultaneously reconstruct 3D face mesh in the world space and predict 2D face landmarks on the image plane to address the problem of perspective 3D face reconstruction. Based on the predicted 3D vertices and 2D landmarks, the 6DoF (6 Degrees of Freedom) face pose can be easily estimated by the PnP solver to represent perspective projection. Our approach achieves 1st place on the leader-board of the ECCV 2022 WCPA challenge and our model is visually robust under different identities, expressions and poses. The training code and models are released to facilitate future research.

Authors

J. Guo

6 papers

Alexandros Lattas

2 papers

Jinke Yu

2 papers

Perspective Reconstruction of Human Faces by Joint Mesh and Landmark Regression

TL;DR

Abstract

Authors

References31 items

WebFace260M: A Benchmark for Million-Scale Deep Face Recognition

Towards Metrical Reconstruction of Human Faces

3D face reconstruction with dense landmarks

Killing Two Birds with One Stone: Efficient and Robust Training of Face Recognition CNNs by Partial FC

Variational Prototype Learning for Deep Face Recognition

Sample and Computation Redistribution for Efficient Face Detection

WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognition

Learning an animatable detailed 3D face model from in-the-wild images

Designing Network Design Spaces

Dense 3D Face Decoding Over 2500FPS: Joint Texture & Shape Convolutional Mesh Decoders

Accurate 3D Face Reconstruction With Weakly-Supervised Learning: From Single Image to Image Set

Joint Face Detection and Facial Motion Retargeting for Multiple Faces

GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction

The Menpo Benchmark for Multi-pose 2D and 3D Facial Landmark Localisation and Tracking

Unsupervised Training for 3D Morphable Model Regression

Nonlinear 3D Face Morphable Model

Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network

ArcFace: Additive Angular Margin Loss for Deep Face Recognition

Large Scale 3D Morphable Models

Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression

How Far are We from Solving the 2D & 3D Face Alignment Problem? (and a Dataset of 230,000 3D Facial Landmarks)

Regressing Robust and Discriminative 3D Morphable Models with a Very Deep Neural Network

DenseReg: Fully Convolutional Dense Shape Regression In-the-Wild

Pose Estimation for Augmented Reality: A Hands-On Survey

Deep Residual Learning for Image Recognition

FaceNet: A unified embedding for face recognition and clustering

A Morphable Model For The Synthesis Of 3D Faces

Single-Image 3D Face Reconstruction under Perspective Projection

Sub-center ArcFace: Boosting Face Recognition by Large-Scale Noisy Web Faces

Face alignment across large poses: A 3D solution

RetinaFace: Single-shot Multi-level Face Localization in the Wild

Field of Study