Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation (2020-02-20T00:00:00.000000Z)

TL;DR

This work proposes a simple yet generic representation learning framework, named SHOT, which freezes the classifier module (hypothesis) of the source model and learns the target-specific feature extraction module by exploiting both information maximization and self-supervised pseudo-labeling to implicitly align representations from the target domains to the source hypothesis.

Abstract

Unsupervised domain adaptation (UDA) aims to leverage the knowledge learned from a labeled source dataset to solve similar tasks in a new unlabeled domain. Prior UDA methods typically require to access the source data when learning to adapt the model, making them risky and inefficient for decentralized private data. This work tackles a practical setting where only a trained source model is available and investigates how we can effectively utilize such a model without source data to solve UDA problems. We propose a simple yet generic representation learning framework, named \emph{Source HypOthesis Transfer} (SHOT). SHOT freezes the classifier module (hypothesis) of the source model and learns the target-specific feature extraction module by exploiting both information maximization and self-supervised pseudo-labeling to implicitly align representations from the target domains to the source hypothesis. To verify its versatility, we evaluate SHOT in a variety of adaptation cases including closed-set, partial-set, and open-set domain adaptation. Experiments indicate that SHOT yields state-of-the-art results among multiple domain adaptation benchmarks.

Authors

Jiashi Feng

18 papers

Jian Liang

2 papers

Dapeng Hu

2 papers

TL;DR

Abstract

Authors

References79 items

Exploring uncertainty in pseudo-label guided unsupervised domain adaptation

Federated Adversarial Domain Adaptation

Drop to Adapt: Learning Discriminative Features for Unsupervised Domain Adaptation

Pseudo-Labeling Curriculum for Unsupervised Domain Adaptation

When Does Label Smoothing Help?

Separate to Adapt: Open Set Domain Adaptation via Progressive Separation

Distant Supervised Centroid Shift: A Simple and Efficient Approach to Visual Domain Adaptation

Transferability vs. Discriminability: Batch Spectral Penalization for Adversarial Domain Adaptation

Aggregating Randomized Clustering-Promoting Invariant Projections for Domain Adaptation

Domain Agnostic Learning with Disentangled Representations

Semi-Supervised Domain Adaptation via Minimax Entropy

Learning to Transfer Examples for Partial Domain Adaptation

Cluster Alignment With a Teacher for Unsupervised Domain Adaptation

Sliced Wasserstein Discrepancy for Unsupervised Domain Adaptation

Towards Federated Learning at Scale: System Design

Moment Matching for Multi-Source Domain Adaptation

Source Free Domain Adaptation Using an Off-the-Shelf Classifier

ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation

Larger Norm More Transferable: An Adaptive Feature Norm Approach for Unsupervised Domain Adaptation

Unsupervised Domain Adaptation for Semantic Segmentation via Class-Balanced Self-training

Deep Clustering for Unsupervised Learning of Visual Features

Learning Semantic Representations for Unsupervised Domain Adaptation

Collaborative and Adversarial Network for Unsupervised Domain Adaptation

Open Set Domain Adaptation by Backpropagation

DeepJDOT: Deep Joint distribution optimal transport for unsupervised domain adaptation

Importance Weighted Adversarial Nets for Partial Domain Adaptation

Domain Adaptive Faster R-CNN for Object Detection in the Wild

Deep Cocktail Network: Multi-source Unsupervised Domain Adaptation with Category Shift

Learning to Adapt Structured Output Space for Semantic Segmentation

A DIRT-T Approach to Unsupervised Domain Adaptation

Maximum Classifier Discrepancy for Unsupervised Domain Adaptation

CyCADA: Cycle-Consistent Adversarial Domain Adaptation

Adversarial Dropout Regularization

Practical Secure Aggregation for Privacy-Preserving Machine Learning

Learning Differentially Private Recurrent Language Models

VisDA: The Visual Domain Adaptation Challenge

Open Set Domain Adaptation

Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes

Partial Transfer Learning with Selective Adversarial Networks

Deep Hashing Network for Unsupervised Domain Adaptation

Self-ensembling for visual domain adaptation

Conditional Adversarial Domain Adaptation

AutoDIAL: Automatic Domain Alignment Layers

Learning Discrete Representations via Information Maximizing Self-Augmented Training

Adversarial Discriminative Domain Adaptation

Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning

Domain Separation Networks

Domain Adaptation in the Absence of Source Domain Data

Deep Reconstruction-Classification Networks for Unsupervised Domain Adaptation

Deep Transfer Learning with Joint Adaptation Networks

Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks

Deep Residual Learning for Image Recognition

Towards Open Set Deep Networks

Return of Frustratingly Easy Domain Adaptation

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Learning Transferable Features with Deep Adaptation Networks

How transferable are features in deep neural networks?

Unsupervised Domain Adaptation by Backpropagation

Learning Categories From Few Examples With Multi Model Knowledge Transfer

Transfer Feature Learning with Joint Distribution Adaptation

Stability and Hypothesis Transfer Learning

Information-Theoretical Learning of Discriminative Clusters for Unsupervised Domain Adaptation

Geodesic flow kernel for unsupervised domain adaptation

Domain Adaptation for Large-Scale Sentiment Classification: A Deep Learning Approach

Discriminative Clustering by Regularized Information Maximization

Adapting Visual Category Models to New Domains

A theory of learning from different domains

Domain Adaptation with Multiple Sources

Instance Weighting for Domain Adaptation in NLP

Cross-domain video concept detection using adaptive svms

Correcting Sample Selection Bias by Unlabeled Data

A Kernel Method for the Two-Sample-Problem

Semi-supervised Learning by Entropy Minimization

Transferable Normalization: Towards Improving Transferability of Deep Neural Networks

GENERATIVE ADVERSARIAL NETS

A Comprehensive Survey on Domain Adaptation for Visual Applications