Meta Discovery: Learning to Discover Novel Classes given Very Limited Data (2021-02-08T00:00:00.000000Z)

TL;DR

This paper demystify assumptions behind NCD and finds that high-level semantic features should be shared among the seen and unseen classes, which proves that NCD is theoretically solvable under certain assumptions and can be naturally linked to meta-learning that has exactly the same assumption as NCD.

Abstract

In novel class discovery (NCD), we are given labeled data from seen classes and unlabeled data from unseen classes, and we train clustering models for the unseen classes. However, the implicit assumptions behind NCD are still unclear. In this paper, we demystify assumptions behind NCD and find that high-level semantic features should be shared among the seen and unseen classes. Based on this finding, NCD is theoretically solvable under certain assumptions and can be naturally linked to meta-learning that has exactly the same assumption as NCD. Thus, we can empirically solve the NCD problem by meta-learning algorithms after slight modifications. This meta-learning-based methodology significantly reduces the amount of unlabeled data needed for training and makes it more practical, as demonstrated in experiments. The use of very limited data is also justified by the application scenario of NCD: since it is unnatural to label only seen-class data, NCD is sampling instead of labeling in causality. Therefore, unseen-class data should be collected on the way of collecting seen-class data, which is why they are novel and first need to be clustered.

Authors

Tongliang Liu

8 papers

Bo Han

6 papers

Gang Niu

11 papers

TL;DR

Abstract

Authors

References79 items

Where and How to Transfer: Knowledge Aggregation-Induced Transferability Perception for Unsupervised Domain Adaptation

A Multi-Mode Modulator for Multi-Domain Few-Shot Classification

A Unified Objective for Novel Class Discovery

The Bottleneck

Learning Bounds for Open-Set Learning

AutoNovel: Automatically Discovering and Learning Novel Visual Categories

TOHAN: A One-step Approach towards Few-shot Hypothesis Adaptation

Neighborhood Contrastive Learning for Novel Class Discovery

Joint Representation Learning and Novel Category Discovery on Single- and Multi-modal Data

Isometric Propagation Network for Generalized Zero-shot Learning

Improving Generalization in Meta-learning via Task Augmentation

Adaptive Task Sampling for Meta-Learning

Laplacian Regularized Few-Shot Learning

Parts-dependent Label Noise: Towards Instance-dependent Label Noise

Rethinking Importance Weighting for Deep Learning under Distribution Shift

What Can Be Transferred: Unsupervised Domain Adaptation for Endoscopic Lesions Segmentation

OpenMix: Reviving Known Knowledge for Discovering Novel Visual Categories in an Open World

Meta-Learning in Neural Networks: A Survey

Attribute Propagation Network for Graph Zero-Shot Learning

DEPARA: Deep Attribution Graph for Deep Knowledge Transferability

Learning Deep Kernels for Non-Parametric Two-Sample Tests

Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation

Adaptive Region-Based Active Learning

Automatically Discovering and Learning New Visual Categories with Ranking Statistics

Dynamic Instance Normalization for Arbitrary Style Transfer

Deep Model Transferability from Attribution Maps

Learning to Discover Novel Visual Categories via Deep Transfer Clustering

Few-Shot Learning With Global Class Representations

Open Set Domain Adaptation: Theoretical Bound and Algorithm

Butterfly: A Panacea for All Difficulties in Wildly Unsupervised Domain Adaptation

Hierarchically Structured Meta-learning

Generalizing from a Few Examples

Adaptive Posterior Learning: few-shot learning with a surprise-based memory module

Multi-class Classification without Multi-class Labels

SIGUA: Forgetting May Make Learning with Noisy Labels More Robust

Unsupervised Learning

Learning to Propagate Labels: Transductive Propagation Network for Few-Shot Learning

Unsupervised Representation Learning by Predicting Image Rotations

Learning to cluster in order to Transfer across domains and tasks

Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theory

Conditional Adversarial Domain Adaptation

Prototypical Networks for Few-shot Learning

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

Positive-Unlabeled Learning with Non-Negative Risk Estimator

Asymmetric Tri-training for Unsupervised Domain Adaptation

Heterogeneous Domain Adaptation: An Unsupervised Approach

Optimization as a Model for Few-Shot Learning

A Survey of Multi-View Representation Learning

Meta-Learning with Memory-Augmented Neural Networks

Domain Adaptation with Conditional Transferable Components

Theoretical Comparisons of Positive-Unlabeled Learning against Positive-Negative Learning

Human-level concept learning through probabilistic program induction

Deep Residual Learning for Image Recognition

Unsupervised Deep Embedding for Clustering Analysis

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Adam: A Method for Stochastic Optimization

Very Deep Convolutional Networks for Large-Scale Image Recognition

Stability and Hypothesis Transfer Learning

ImageNet: A large-scale hierarchical image database

Learning classifiers from only positive and unlabeled data

Algorithmic Stability and Meta-Learning

Learning from Positive and Unlabeled Examples with Different Data Distributions

Discovery of Remains of Plants and Insects

Confident Anchor-Induced Multi-Source Free Domain Adaptation

Meta-learning Hyperparameter Performance Prediction with Neural Processes

A Closer Look at the Training Strategy for Modern Meta-Learning

MATE: Plugging in Model Awareness to Task Embedding for Meta Learning

Multi-class Heterogeneous Domain Adaptation

Transfer Learning via Minimizing the Performance Gap Between Domains

TAEML: Task-Adaptive Ensemble of Meta-Learners

Dropout: a simple way to prevent neural networks from overfitting

Reading Digits in Natural Images with Unsupervised Feature Learning

Learning Multiple Layers of Features from Tiny Images

Some methods for classification and analysis of multivariate observations

Ieee Transactions on Pattern Analysis and Machine Intelligence Classification with Noisy Labels by Importance Reweighting

Each block comprises a 64-filter