Generative Hierarchical Features from Synthesizing Images (2020-07-20T00:00:00.000000Z)

TL;DR

This work considers the pre-trained StyleGAN generator as a learned loss function and utilizes its layer-wise representation to train a novel hierarchical encoder, termed as Generative Hierarchical Feature (GH-Feat), which has strong transferability to both generative and discriminative tasks.

Abstract

Generative Adversarial Networks (GANs) have recently advanced image synthesis by learning the underlying distribution of the observed data. However, how the features learned from solving the task of image generation are applicable to other vision tasks remains seldom explored. In this work, we show that learning to synthesize images can bring remarkable hierarchical visual features that are generalizable across a wide range of applications. Specifically, we consider the pre-trained StyleGAN generator as a learned loss function and utilize its layer-wise representation to train a novel hierarchical encoder. The visual feature produced by our encoder, termed as Generative Hierarchical Feature (GH-Feat), has strong transferability to both generative and discriminative tasks, including image editing, image harmonization, image classification, face verification, landmark detection, and layout prediction. Extensive qualitative and quantitative experimental results demonstrate the appealing performance of GH-Feat.1

Authors

Bolei Zhou

20 papers

Ceyuan Yang

5 papers

Yinghao Xu

4 papers

TL;DR

Abstract

Authors

References74 items

What makes instance discrimination good for transfer learning?

InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs

Adversarial Latent Autoencoders

Temporal Pyramid Network for Action Recognition

In-Domain GAN Inversion for Real Image Editing

Semantic Pyramid for Image Generation

Image Processing Using Multi-Code GAN Prior

Momentum Contrast for Unsupervised Visual Representation Learning

Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesis

LIA: Latently Invertible Autoencoder with Adversarial Learning

On the "steerability" of generative adversarial networks

Large Scale Adversarial Representation Learning

Disentangled Inference for GANs With Latently Invertible Autoencoder

Contrastive Multiview Coding

Data-Efficient Image Recognition with Contrastive Predictive Coding

Local Aggregation for Unsupervised Learning of Visual Embeddings

Edge-Semantic Learning Strategy for Layout Estimation in Indoor Environment

A Style-Based Generator Architecture for Generative Adversarial Networks

Self-Supervised GANs via Auxiliary Rotation Loss

Large Scale GAN Training for High Fidelity Natural Image Synthesis

GAN Dissection: Visualizing and Understanding Generative Adversarial Networks

Learning deep representations by mutual information estimation and maximization

Representation Learning with Contrastive Predictive Coding

Unsupervised Learning of Object Landmarks through Conditional Image Generation

Places: A 10 Million Image Database for Scene Recognition

Unsupervised Feature Learning via Non-parametric Instance Discrimination

LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image

Path Aggregation Network for Instance Segmentation

Unsupervised Representation Learning by Predicting Image Rotations

Progressive Growing of GANs for Improved Quality, Stability, and Variation

Multi-task Self-Supervised Visual Learning

GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium

Network Dissection: Quantifying Interpretability of Deep Visual Representations

Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization

Learning Features by Watching Objects Move

Feature Pyramid Networks for Object Detection

Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction

Adversarially Learned Inference

Adversarial Feature Learning

Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks

Colorful Image Colorization

Perceptual Losses for Real-Time Style Transfer and Super-Resolution

Autoencoding beyond pixels using a learned similarity metric

Deep Residual Learning for Image Recognition

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop

Unsupervised Visual Representation Learning by Context Prediction

U-Net: Convolutional Networks for Biomedical Image Segmentation

Adam: A Method for Stochastic Optimization

Object Detectors Emerge in Deep Scene CNNs

Deep Learning Face Attributes in the Wild

How transferable are features in deep neural networks?

Going deeper with convolutions

Facial Landmark Detection by Deep Multi-task Learning

Very Deep Convolutional Networks for Large-Scale Image Recognition

Discriminative Unsupervised Feature Learning with Convolutional Neural Networks

Microsoft COCO: Common Objects in Context

CNN Features Off-the-Shelf: An Astounding Baseline for Recognition

Visualizing and Understanding Convolutional Neural Networks

ImageNet classification with deep convolutional neural networks

Representation Learning: A Review and New Perspectives

ImageNet: A large-scale hierarchical image database

Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments

SURF: Speeded Up Robust Features

Histograms of oriented gradients for human detection

Distinctive Image Features from Scale-Invariant Keypoints

Image quality assessment: from error visibility to structural similarity

AUTO-ENCODING VARIATIONAL BAYES

Pyotrch cifar10. https://github.com/ kuangliu/pytorch-cifar.git, 2019

GENERATIVE ADVERSARIAL NETS

Distinctive Image Features from Scale-Invariant Keypoints Abstract by Matthijs Dorst Based on the paper by

Supporting Online Material for Reducing the Dimensionality of Data with Neural Networks

Gradient-based learning applied to document recognition

Stylegan -official tensorflow implementation

Field of Study

Journal Information