SfSNet: Learning Shape, Reflectance and Illuminance of Faces 'in the Wild' (2017-12-02T00:00:00.000000Z)

TL;DR

SfSNet produces significantly better quantitative and qualitative results than state-of-the-art methods for inverse rendering and independent normal and illumination estimation and is designed to reflect a physical lambertian rendering model.

Abstract

We present SfSNet, an end-to-end learning framework for producing an accurate decomposition of an unconstrained human face image into shape, reflectance and illuminance. SfSNet is designed to reflect a physical lambertian rendering model. SfSNet learns from a mixture of labeled synthetic and unlabeled real world images. This allows the network to capture low frequency variations from synthetic and high frequency details from real images through the photometric reconstruction loss. SfSNet consists of a new decomposition architecture with residual blocks that learns a complete separation of albedo and normal. This is used along with the original image to predict lighting. SfSNet produces significantly better quantitative and qualitative results than state-of-the-art methods for inverse rendering and independent normal and illumination estimation.

Authors

Angjoo Kanazawa

16 papers

C. Castillo

3 papers

Soumyadip Sengupta

4 papers

TL;DR

Abstract

Authors

References44 items

Label Denoising Adversarial Network (LDAN) for Inverse Lighting of Faces

Self-Supervised Intrinsic Image Decomposition

Adaptive 3D Face Reconstruction from Unconstrained Photo Collections

Label Denoising Adversarial Network (LDAN) for Inverse Lighting of Face Images

Neural Face Editing with Intrinsic Image Disentangling

MoFA: Model-Based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction

Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks

Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translation

Unsupervised Image-to-Image Translation Networks

Learning Non-Lambertian Object Intrinsics Across ShapeNet Categories

Regressing Robust and Discriminative 3D Morphable Models with a Very Deep Neural Network

Photorealistic Facial Texture Inference Using Deep Neural Networks

Image-to-Image Translation with Conditional Adversarial Networks

An All-In-One Convolutional Neural Network for Face Analysis

Production-level facial performance capture using deep convolutional neural networks

UberNet: Training a Universal Convolutional Neural Network for Low-, Mid-, and High-Level Vision Using Diverse Datasets and Limited Memory

Face2Face: Real-Time Face Capture and Reenactment of RGB Videos

Real-Time Facial Segmentation and Performance Capture from RGB Input

Marr Revisited: 2D-3D Alignment via Surface Normal Prediction

Perceptual Losses for Real-Time Style Transfer and Super-Resolution

Identity Mappings in Deep Residual Networks

ShapeNet: An Information-Rich 3D Model Repository

Direct Intrinsics: Learning Albedo-Shading Decomposition by Convolutional Regression

High-quality hair modeling from a single portrait photo

U-Net: Convolutional Networks for Biomedical Image Segmentation

Deep Convolutional Inverse Graphics Network

Deep Learning Face Attributes in the Wild

Intrinsic images in the wild

Shape and Reflectance from Natural Illumination

Deep Lambertian Networks

Face reconstruction in the wild

The Photoface database

Face Relighting from a Single Image under Arbitrary Unknown Lighting Conditions

Et al

Multi-PIE

Handbook of Mathematical Models in Computer Vision

Recovering intrinsic images from a single image

Lambertian reflectance and linear subspaces

A Morphable Model For The Synthesis Of 3D Faces

Shape

Normal Estimation For ”in-the-wild” Faces Using Fully Convolutional Networks

Shape From Shading

Ieee Transactions on Pattern Analysis and Machine Intelligence 1 3d Face Reconstruction from a Single Image Using a Single Reference Face Shape

Ieee Transactions on Pattern Analysis and Machine Intelligence Shape, Illumination, and Reflectance from Shading

Field of Study

Journal Information

Name

Page