Rethinking Clustering-Based Pseudo-Labeling for Unsupervised Meta-Learning (2022-09-27T00:00:00.000000Z)

TL;DR

This work addresses the core reason for lack of a clustering-friendly property in the embedding space by minimizing the inter- to intra-class similarity ratio to provide clustered-friendly embedding features, and validates the approach through comprehensive experiments.

Abstract

The pioneering method for unsupervised meta-learning, CACTUs, is a clustering-based approach with pseudo-labeling. This approach is model-agnostic and can be combined with supervised algorithms to learn from unlabeled data. However, it often suffers from label inconsistency or limited diversity, which leads to poor performance. In this work, we prove that the core reason for this is lack of a clustering-friendly property in the embedding space. We address this by minimizing the inter- to intra-class similarity ratio to provide clustering-friendly embedding features, and validate our approach through comprehensive experiments. Note that, despite only utilizing a simple clustering algorithm (k-means) in our embedding space to obtain the pseudo-labels, we achieve significant improvement. Moreover, we adopt a progressive evaluation mechanism to obtain more diverse samples in order to further alleviate the limited diversity problem. Finally, our approach is also model-agnostic and can easily be integrated into existing supervised methods. To demonstrate its generalization ability, we integrate it into two representative algorithms: MAML and EP. The results on three main few-shot benchmarks clearly show that the proposed method achieves significant improvement compared to state-of-the-art models. Notably, our approach also outperforms the corresponding supervised method in two tasks.

Authors

Jianbing Shen

16 papers

Ling Shao

32 papers

Xingping Dong

1 papers

TL;DR

Abstract

Authors

References57 items

ScaleNet: An Unsupervised Representation Learning Method for Limited Information

Multi-Level Representation Learning with Semantic Alignment for Referring Video Object Segmentation

Learning to Fuse Asymmetric Feature Maps in Siamese Trackers

Unsupervised Few-Shot Feature Learning via Self-Supervised Training

Inferring Salient Objects from Human Fixations

Self-Supervised Prototypical Transfer Learning for Few-Shot Classification

Unsupervised Meta-Learning through Latent-Space Interpolation in Generative Models

Unsupervised Few-shot Learning via Distribution Shift-based Augmentation

Improved Baselines with Momentum Contrastive Learning

Embedding Propagation: Smoother Manifold for Few-Shot Classification

PyTorch: An Imperative Style, High-Performance Deep Learning Library

Momentum Contrast for Unsupervised Visual Representation Learning

Few-Shot Image Recognition With Knowledge Transfer

Distilled Siamese Networks for Visual Tracking

Learning to Self-Train for Semi-Supervised Few-Shot Classification

Few-Shot Learning With Embedded Class Models and Shot-Free Meta Training

A Closer Look at Few-shot Classification

Diversity With Cooperation: Ensemble Methods for Few-Shot Classification

Assume, Augment and Learn: Unsupervised Few-Shot Meta-Learning via Random Labels and Data Augmentation

Unsupervised Meta-Learning for Few-Shot Image Classification

Unsupervised Learning via Meta-Learning

Understanding and Improving Interpolation in Autoencoders via an Adversarial Regularizer

Probabilistic Model-Agnostic Meta-Learning

Unsupervised Feature Learning via Non-parametric Instance Discrimination

Meta-Learning for Semi-Supervised Few-Shot Classification

Universal Language Model Fine-tuning for Text Classification

Prototypical Networks for Few-shot Learning

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction

Unsupervised Pretraining for Sequence to Sequence Learning

Optimization as a Model for Few-Shot Learning

Using Fast Weights to Attend to the Recent Past

Meta-Learning with Memory-Augmented Neural Networks

Learning to learn by gradient descent by gradient descent

Matching Networks for One Shot Learning

InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets

Adversarial Feature Learning

Sub-Markov Random Walk for Image Segmentation

Semi-supervised Sequence Learning

Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

Roles of Pre-Training and Fine-Tuning in Context-Dependent DBN-HMMs for Real-World Speech Recognition

Why Does Unsupervised Pre-training Help Deep Learning?

Extracting and composing robust features with denoising autoencoders

Greedy Layer-Wise Training of Deep Networks

Efficient Learning of Sparse Representations with an Energy-Based Model

A Fast Learning Algorithm for Deep Belief Nets

Learning to Learn: Introduction and Overview

Meta-GMVAE: Mixture of Gaussian VAE for Unsupervised Meta-Learning

CLNet: A Compact Latent Network for Fast Adjusting Siamese Trackers

Improving Language Understanding by Generative Pre-Training

Siamese Neural Networks for One-Shot Image Recognition

One shot learning of simple visual concepts

Using fast weights to deblur old memories

Evolutionary principles in self-referential learning, or on learning how to learn: the meta-meta-.

Learning curves for stochastic gradient descent in linear feedforward networks

MAML (supervised

Rethinking Clustering-Based Pseudo-Labeling for UML 17

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names