REALY: Rethinki… (2022-03-18T00:00:00.000000Z)

TL;DR

A novel evaluation approach with a new benchmark REALY, consists of 100 globally aligned face scans with accurate facial keypoints, high-quality region masks, and topology-consistent meshes that performs region-wise shape alignment and leads to more accurate, bidirectional correspondences during computing the shape errors.

Abstract

The evaluation of 3D face reconstruction results typically relies on a rigid shape alignment between the estimated 3D model and the ground-truth scan. We observe that aligning two shapes with different reference points can largely affect the evaluation results. This poses difficulties for precisely diagnosing and improving a 3D face reconstruction method. In this paper, we propose a novel evaluation approach with a new benchmark REALY, consists of 100 globally aligned face scans with accurate facial keypoints, high-quality region masks, and topology-consistent meshes. Our approach performs region-wise shape alignment and leads to more accurate, bidirectional correspondences during computing the shape errors. The fine-grained, region-wise evaluation results provide us detailed understandings about the performance of state-of-the-art 3D face reconstruction methods. For example, our experiments on single-image based reconstruction methods reveal that DECA performs the best on nose regions, while GANFit performs better on cheek regions. Besides, a new and high-quality 3DMM basis, HIFI3D++, is further derived using the same procedure as we construct REALY to align and retopologize several 3D face datasets. We will release REALY, HIFI3D++, and our new evaluation pipeline at https://realy3dface.com.

Authors

Jing Ren

2 papers

Zenghao Chai

1 papers

Haoxian Zhang

2 papers

References81 items

Self-Supervised 3D Face Reconstruction via Conditional Estimation

H3D-Net: Few-Shot High-Fidelity 3D Head Reconstruction

Normalized Avatar Synthesis Using StyleGAN and Perceptual Refinement

Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collection

Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstruction

Abstract

References81 items

TL;DR

Abstract

Authors

References81 items

Self-Supervised 3D Face Reconstruction via Conditional Estimation

H3D-Net: Few-Shot High-Fidelity 3D Head Reconstruction

Normalized Avatar Synthesis Using StyleGAN and Perceptual Refinement

Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collection

Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstruction

TL;DR

Abstract

Authors

References81 items

Self-Supervised 3D Face Reconstruction via Conditional Estimation

H3D-Net: Few-Shot High-Fidelity 3D Head Reconstruction

Normalized Avatar Synthesis Using StyleGAN and Perceptual Refinement

Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collection

Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstruction

Inverting Generative Adversarial Renderer for Face Reconstruction

Pixel Codec Avatars

Reconstructing Recognizable 3D Face Shapes based on 3D Morphable Models

Riggable 3D Face Reconstruction via In-Network Optimization

Towards High Fidelity Monocular Face Reconstruction with Rich Reflectance using Self-supervised Learning and Ray Tracing

MeInGame: Create a Game Character Face from a Single Portrait

i3DMM: Deep Implicit 3D Morphable Model of Human Heads

Do 2D GANs Know 3D Shape? Unsupervised 3D shape reconstruction from 2D Image GANs

High-Fidelity 3D Digital Human Head Creation from RGB-D Selfies

Learning 3D Face Reconstruction with a Pose Guidance Network

Learning Complete 3D Morphable Face Models from Images and Videos

Pixel-Face: A Large-Scale, High-Resolution Benchmark for 3D Face Reconstruction

Towards Fast, Accurate and Stable 3D Dense Face Alignment

Self-Supervised Monocular 3D Face Reconstruction by Occlusion-Aware Multi-view Geometry Consistency

Personalized Face Modeling for Improved Face Reconstruction and Motion Retargeting

Uncertainty-Aware Mesh Decoder for High Fidelity 3D Face Reconstruction

Semi-supervised 3D Face Representation Learning from Unconstrained Photo Collections

A Morphable Face Albedo Model

FaceScape: A Large-Scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction

AvatarMe: Realistically Renderable 3D Facial Reconstruction “In-the-Wild”

Towards High-Fidelity 3D Face Reconstruction From In-the-Wild Images Using Graph Convolutional Networks

Unsupervised Learning of Probably Symmetric Deformable 3D Objects From Images in the Wild

Statistical Modeling of Craniofacial Shape and Texture

Self-Supervised Learning of Detailed 3D Face Reconstruction

DF2Net: A Dense-Fine-Finer Network for Detailed 3D Face Reconstruction

3D Morphable Face Models—Past, Present, and Future

Learning to Regress 3D Face Shape and Expression From an Image Without 3D Supervision

Towards High-Fidelity Nonlinear 3D Face Morphable Model

MVF-Net: Multi-View 3D Face Morphable Model Regression

Accurate 3D Face Reconstruction With Weakly-Supervised Learning: From Single Image to Image Set

Combining 3D Morphable Models: A Large Scale Face-And-Head Model

GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction

A Style-Based Generator Architecture for Generative Adversarial Networks

On Learning 3D Face Morphable Model from In-the-Wild Images

High-fidelity facial reflectance and geometry inference from an unconstrained image

Unsupervised Training for 3D Morphable Model Regression

State of the Art on Monocular 3D Face Reconstruction, Tracking, and Applications

Nonlinear 3D Face Morphable Model

Disentangling Features in 3D Face Shapes for Joint Face Reconstruction and Recognition

Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network

Evaluation of Dense 3D Reconstruction from 2D Face Images in the Wild

Pose-Robust Face Recognition via Deep Residual Equivariant Mapping

ExpNet: Landmark-Free, Deep, 3D Facial Expressions

Learning a model of facial shape and expression from 4D scans

Avatar digitization from a single image for real-time rendering

FacePoseNet: Making a Case for Landmark-Free Face Alignment

MoFA: Model-Based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction

Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translation

Regressing Robust and Discriminative 3D Morphable Models with a Very Deep Neural Network

3D Face Reconstruction by Learning from Synthetic Data

Real-time facial animation with image-based dynamic avatars

Face2Face: Real-Time Face Capture and Reenactment of RGB Videos

A 3D Morphable Model Learnt from 10,000 Faces

Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks

Face Alignment in Full Pose Range: A 3D Total Solution

Adam: A Method for Stochastic Optimization

Deep Learning Face Attributes in the Wild

FaceWarehouse: A 3D Facial Expression Database for Visual Computing

3D shape regression for real-time facial animation

Review of statistical shape spaces for 3D data with comparative analysis for human faces

Face detection, pose estimation, and landmark localization in the wild

The florence 2D/3D hybrid face dataset