3D Human Texture Estimation from a Single Image with Transformers

Published in

IEEE International Conference on Computer Vision(2021)

External Links:

Generate Graph DownloadPDF

TL;DR

A Transformer-based framework for 3D human texture estimation from a single image is proposed, able to effectively exploit the global information of the input image, overcoming the limitations of existing methods that are solely based on convolutional neural networks.

Abstract

We propose a Transformer-based framework for 3D human texture estimation from a single image. The proposed Transformer is able to effectively exploit the global information of the input image, overcoming the limitations of existing methods that are solely based on convolutional neural networks. In addition, we also propose a mask-fusion strategy to combine the advantages of the RGB-based and texture-flow-based models. We further introduce a part-style loss to help reconstruct high-fidelity colors without introducing unpleasant artifacts. Extensive experiments demonstrate the effectiveness of the proposed method against state-of-the-art 3D human texture estimation approaches both quantitatively and qualitatively. The project page is at https://www.mmlab-ntu.com/project/texformer.

Authors

Chen Change Loy

72 papers

References59 items

SMPL: A Skinned Multi-Person Linear Model

3D Human Pose, Shape and Texture From Low-Resolution Images and Videos

Pre-Trained Image Processing Transformer

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

TexMesh: Reconstructing Detailed Human Texture and Geometry from RGB-D Video

3D Human Texture Estimation from a Single Image with Transformers

Published in

IEEE International Conference on Computer Vision(2021)

External Links:

Generate Graph DownloadPDF

TL;DR

Abstract

Authors

Chen Change Loy

72 papers

References59 items

SMPL: A Skinned Multi-Person Linear Model

3D Human Pose, Shape and Texture From Low-Resolution Images and Videos

Pre-Trained Image Processing Transformer

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

TexMesh: Reconstructing Detailed Human Texture and Geometry from RGB-D Video

Reconstructing NBA Players

3D Human Shape and Pose from a Single Low-Resolution Image with Self-Supervised Learning

Shape and Viewpoint without Keypoints

PaMIR: Parametric Model-Conditioned Implicit Representation for Image-Based Human Reconstruction

Learning Spatial and Spatio-Temporal Pixel Aggregations for Image and Video Denoising

End-to-End Object Detection with Transformers

ARCH: Animatable Reconstruction of Clothed Humans

Robust 3D Self-Portraits in Seconds

PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization

Self-supervised Single-view 3D Reconstruction via Semantic Consistency

Learning to Transfer Texture From Clothing Images to 3D Humans

VIBE: Video Inference for Human Body Pose and Shape Estimation

Quadratic video interpolation

Torchreid: A Library for Deep Learning Person Re-Identification in Pytorch

Learning to Reconstruct 3D Human Pose and Shape via Model-Fitting in the Loop

360-Degree Textures of People in Clothing from a Single Image

Multi-Garment Net: Learning to Dress 3D People From Images

PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization

Tex2Shape: Detailed Full Human Body Geometry From a Single Image

3DPeople: Modeling the Geometry of Dressed Humans

Re-Identification Supervised Texture Generation

DeepHuman: 3D Human Reconstruction From a Single Image

Learning to Reconstruct People in Clothing From a Single RGB Camera

SiCloPe: Silhouette-Based Clothed People

EANet: Enhancing Alignment for Cross-Domain Person Re-identification

Learning 3D Human Dynamics From Video

Deep Volumetric Video From Very Sparse Multi-view Performance Capture

Dense Pose Transfer

Detailed Human Avatars from Monocular Video

Learning Category-Specific Mesh Reconstruction from Image Collections

Video Based Reconstruction of 3D People Models

The Unreasonable Effectiveness of Deep Features as a Perceptual Metric

End-to-End Recovery of Human Shape and Pose

Beyond Part Models: Person Retrieval with Refined Part Pooling

Neural 3D Mesh Renderer

Learning to Super-Resolve Blurry Face and Text Images

Attention is All you Need

Pose Guided Person Image Generation

Learning from Synthetic Humans

Image-to-Image Translation with Conditional Adversarial Networks

Layer Normalization

Image Style Transfer Using Convolutional Neural Networks

Deep Residual Learning for Image Recognition

Scalable Person Re-identification: A Benchmark

U-Net: Convolutional Networks for Biomedical Image Segmentation

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Image quality assessment: from error visibility to structural similarity

Fast approximate energy minimization via graph cuts

Analysis of approximate nearest neighbor searching with clustered point sets

Learning Factorized Weight Matrix for Joint Filtering

Human Parsing Based Texture Transfer from Single Image to 3D Human via Cross-View Consistency

Hand-Transformer: Non-Autoregressive Structured Modeling for 3D Hand Pose Estimation

A survey on visual transformer

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Field of Study

Computer Science

Journal Information

Name

2021 IEEE/CVF International Conference on Computer Vision (ICCV)

Page

13829-13838

Venue Information

Name

IEEE International Conference on Computer Vision

Type

conference

URL

https://ieeexplore.ieee.org/xpl/conhome/1000149/all-proceedings

Alternate Names

ICCV
IEEE Int Conf Comput Vis
ICCV Workshops
ICCV Work