Unsupervised Domain Expansion for Visual Categorization (2021-04-01T00:00:00.000000Z)

TL;DR

This article proposes a new task called unsupervised domain expansion (UDE), which aims to adapt a deep model for the target domain with its unlabeled data, meanwhile maintaining the model’s performance on the source domain.

Abstract

Expanding visual categorization into a novel domain without the need of extra annotation has been a long-term interest for multimedia intelligence. Previously, this challenge has been approached by unsupervised domain adaptation (UDA). Given labeled data from a source domain and unlabeled data from a target domain, UDA seeks for a deep representation that is both discriminative and domain-invariant. While UDA focuses on the target domain, we argue that the performance on both source and target domains matters, as in practice which domain a test example comes from is unknown. In this article, we extend UDA by proposing a new task called unsupervised domain expansion (UDE), which aims to adapt a deep model for the target domain with its unlabeled data, meanwhile maintaining the model’s performance on the source domain. We propose Knowledge Distillation Domain Expansion (KDDE) as a general method for the UDE task. Its domain-adaptation module can be instantiated with any existing model. We develop a knowledge distillation-based learning mechanism, enabling KDDE to optimize a single objective wherein the source and target domains are equally treated. Extensive experiments on two major benchmarks, i.e., Office-Home and DomainNet, show that KDDE compares favorably against four competitive baselines, i.e., DDC, DANN, DAAN, and CDAN, for both UDA and UDE tasks. Our study also reveals that the current UDA models improve their performance on the target domain at the cost of noticeable performance loss on the source domain.

Authors

Gang Yang

6 papers

Dayong Ding

2 papers

Jie Wang

1 papers

TL;DR

Abstract

Authors

References48 items

Unsupervised Domain Adaptation in Semantic Segmentation: a Review

PyTorch: An Imperative Style, High-Performance Deep Learning Library

Domain Adaptation via Teacher-Student Learning for End-to-End Speech Recognition

Contrastive Representation Distillation

Joint Adversarial Domain Adaptation

Training Efficient Saliency Prediction Models with Knowledge Distillation

Transfer Learning with Dynamic Adversarial Adaptation Network

Confidence Regularized Self-Training

Distilling Knowledge From a Deep Pose Regressor Network

Two-Stream CNN with Loose Pair Training for Multi-modal AMD Categorization

Transferable Attention for Domain Adaptation

Adversarially Robust Distillation

Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation

Semi-Supervised Domain Adaptation via Minimax Entropy

Knowledge Distillation for Recurrent Neural Network Language Modeling with Trust Regularization

Structured Knowledge Distillation for Semantic Segmentation

Improved Knowledge Distillation via Teacher Assistant

Contrastive Adaptation Network for Unsupervised Domain Adaptation

Moment Matching for Multi-Source Domain Adaptation

Unsupervised Domain Adaptation for Semantic Segmentation via Class-Balanced Self-training

Clinically applicable deep learning for diagnosis and referral in retinal disease

Adversarial Teacher-Student Learning for Unsupervised Domain Adaptation

Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning

Maximum Classifier Discrepancy for Unsupervised Domain Adaptation

Learning Efficient Object Detection Models with Knowledge Distillation

Unsupervised Domain Adaptation with Similarity Learning

Examining CNN representations with respect to Dataset Bias

Deep Hashing Network for Unsupervised Domain Adaptation

Self-ensembling for visual domain adaptation

Deep Mutual Learning

Conditional Adversarial Domain Adaptation

Domain adaptation of DNN acoustic models using knowledge distillation

Adversarial Discriminative Domain Adaptation

Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization

Deep CORAL: Correlation Alignment for Deep Domain Adaptation

Deep Transfer Learning with Joint Adaptation Networks

Domain-Adversarial Training of Neural Networks

Distilling the Knowledge in a Neural Network

Learning Transferable Features with Deep Adaptation Networks

FitNets: Hints for Thin Deep Nets

Deep Domain Confusion: Maximizing for Domain Invariance

Domain Transfer SVM for video concept detection

Cross-domain learning methods for high-level visual concept classification

Cross-domain video concept detection using adaptive svms

Analysis of Representations for Domain Adaptation

Transferable Normalization: Towards Improving Transferability of Deep Neural Networks

Visualizing Data using t-SNE

Unsupervised Domain Expansion

Field of Study

Journal Information

Name

Page

Volume