Unsupervised Discovery of Object Landmarks as Structural Representations (2018-04-12T00:00:00.000000Z)

TL;DR

This paper proposes an autoencoding formulation to discover landmarks as explicit structural representations, which naturally creates an unsupervised, perceptible interface to manipulate object shapes and decode images with controllable structures.

Abstract

Deep neural networks can model images with rich latent representations, but they cannot naturally conceptualize structures of object categories in a human-perceptible way. This paper addresses the problem of learning object structures in an image modeling process without supervision. We propose an autoencoding formulation to discover landmarks as explicit structural representations. The encoding module outputs landmark coordinates, whose validity is ensured by constraints that reflect the necessary properties for landmarks. The decoding module takes the landmarks as a part of the learnable input representations in an end-to-end differentiable framework. Our discovered landmarks are semantically meaningful and more predictive of manually annotated landmarks than those discovered by previous methods. The coordinates of our landmarks are also complementary features to pretrained deep-neural-network representations in recognizing visual attributes. In addition, the proposed method naturally creates an unsupervised, perceptible interface to manipulate object shapes and decode images with controllable structures.

Authors

Honglak Lee

21 papers

Y. Zhang

2 papers

Yijie Guo

3 papers

TL;DR

Abstract

Authors

References86 items

Recurrent 3D-2D Dual Learning for Large-Pose Facial Landmark Detection

A Deep Regression Architecture with Two-Stage Re-initialization for High Performance Facial Landmark Detection

Fine-Grained Recognition as HSnet Search for Informative Image Parts

Simultaneous Facial Landmark Detection, Pose and Deformation Estimation Under Facial Occlusion

Unsupervised learning of object frames by dense equivariant image labelling

Unsupervised object learning from dense equivariant image labelling

Unsupervised Learning of Object Landmarks by Factorized Spatial Embeddings

Network Dissection: Quantifying Interpretability of Deep Visual Representations

Learning to Generate Long-term Future via Hierarchical Prediction

AnchorNet: A Weakly Supervised Network to Learn Geometry-Sensitive Features for Semantic Matching

Unsupervised Part Learning for Visual Recognition

BEGAN: Boundary Equilibrium Generative Adversarial Networks

Mask R-CNN

Convolutional Neural Network Architecture for Geometric Matching

Parallel Multiscale Autoregressive Density Estimation

A Point Set Generation Network for 3D Object Reconstruction from a Single Image

Disentangling factors of variation in deep representation using adversarial training

Learning What and Where to Draw

Robust Facial Landmark Detection via Recurrent Attentive-Refinement Networks

End-to-End Learning of Deformable Mixture of Parts and Deep Convolutional Neural Networks for Human Pose Estimation

Conditional Image Generation with PixelCNN Decoders

Learning Covariant Feature Detectors

Deep Deformation Network for Object Landmark Localization

Single Image 3D Interpreter Network

WarpNet: Weakly Supervised Matching for Single-View Reconstruction

Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks

Stacked Hourglass Networks for Human Pose Estimation

Generative Image Modeling Using Style and Structure Adversarial Networks

Pixel Recurrent Neural Networks

Deep Residual Learning for Image Recognition

Deep Visual Analogy-Making

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

Spatial Transformer Networks

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

Learning Deconvolution Network for Semantic Segmentation

Improving object detection with deep convolutional networks via Bayesian optimization and structured prediction

FaceNet: A unified embedding for face recognition and clustering

Unsupervised object discovery and localization in the wild: Part-based matching with bottom-up region proposals

Deep Learning Face Attributes in the Wild

Fully convolutional networks for semantic segmentation

Going deeper with convolutions

Facial Landmark Detection by Deep Multi-task Learning

Coarse-to-Fine Auto-Encoder Networks (CFAN) for Real-Time Face Alignment

Very Deep Convolutional Networks for Large-Scale Image Recognition

Learning Deep Representation for Face Alignment with Auxiliary Attributes

Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments

Face Alignment at 3000 FPS via Regressing Local Binary Features

Fine-Grained Visual Comparisons with Local Learning

Using a Deformation Field Model for Localizing Faces and Facial Points under Weak Supervision

Co-localization in Real-World Images

Learning to Disentangle Factors of Variation with Manifold Interaction

Depth Map Prediction from a Single Image using a Multi-Scale Deep Network

Beyond PASCAL: A benchmark for 3D object detection in the wild

Auto-Encoding Variational Bayes

DeepPose: Human Pose Estimation via Deep Neural Networks

Robust Face Landmark Estimation under Occlusion

Visualizing and Understanding Convolutional Networks

Deep Convolutional Network Cascade for Facial Point Detection

ImageNet classification with deep convolutional neural networks

Face Alignment by Explicit Shape Regression

Real-time facial feature detection using conditional regression forests

Face detection, pose estimation, and landmark localization in the wild

Unsupervised Discovery of Mid-Level Discriminative Patches

Latent structured models for human pose estimation

Annotated Facial Landmarks in the Wild: A large-scale, real-world database for facial landmark localization

Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction

Object Detection with Discriminatively Trained Part Based Models

Facial point detection using boosted regression and graph models

Learning to detect unseen object classes by between-class attribute transfer

A Novel Connectionist System for Unconstrained Handwriting Recognition

Cat Head Detection - How to Effectively Exploit Shape and Texture Features

Automatic feature localisation with constrained local models

Greedy Layer-Wise Training of Deep Networks

Active Appearance Models Revisited

Object class recognition by unsupervised scale-invariant learning

Towards automatic discovery of object categories