SemiReward: A General Reward Model for Semi-supervised Learning (2023-10-04T00:00:00.000000Z)

TL;DR

A Semi-supervised Reward framework (SemiReward) that predicts reward scores to evaluate and filter out high-quality pseudo labels, which is pluggable to mainstream SSL methods in wide task types and scenarios and mitigate confirmation bias is proposed.

Abstract

Semi-supervised learning (SSL) has witnessed great progress with various improvements in the self-training framework with pseudo labeling. The main challenge is how to distinguish high-quality pseudo labels against the confirmation bias. However, existing pseudo-label selection strategies are limited to pre-defined schemes or complex hand-crafted policies specially designed for classification, failing to achieve high-quality labels, fast convergence, and task versatility simultaneously. To these ends, we propose a Semi-supervised Reward framework (SemiReward) that predicts reward scores to evaluate and filter out high-quality pseudo labels, which is pluggable to mainstream SSL methods in wide task types and scenarios. To mitigate confirmation bias, SemiReward is trained online in two stages with a generator model and subsampling strategy. With classification and regression tasks on 13 standard SSL benchmarks across three modalities, extensive experiments verify that SemiReward achieves significant performance gains and faster convergence speeds upon Pseudo Label, FlexMatch, and Free/SoftMatch. Code and models are available at https://github.com/Westlake-AI/SemiReward.

Authors

Siyuan Li

4 papers

Stan Z. Li

2 papers

Zedong Wang

2 papers

TL;DR

Abstract

Authors

References81 items

Shrinking Class Space for Enhanced Certainty in Semi-Supervised Learning

SoftMatch: Addressing the Quantity-Quality Trade-off in Semi-supervised Learning

C-Mixup: Improving Generalization in Regression

ConMatch: Semi-Supervised Learning with Confidence-Guided Consistency Regularization

USB: A Unified Semi-supervised Learning Benchmark

Hyperspherical Consistency Regularization

FreeMatch: Self-adaptive Thresholding for Semi-supervised Learning

Harnessing Hard Mixed Samples with Decoupled Regularizer

SimMatch: Semi-supervised Learning with Similarity Matching

Training language models to follow instructions with human feedback

Debiased Self-Training for Semi-Supervised Learning

Revisiting Consistency Regularization for Semi-Supervised Learning

FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling

Dash: Semi-Supervised Learning with Dynamic Thresholding

Co-learning: Learning from Noisy Labels with Self-supervision

The Bottleneck

HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units

AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation

All Tokens Matter: Token Labeling for Training Better Vision Transformers

The Semi-Supervised iNaturalist-Aves Challenge at FGVC7 Workshop

Self-Tuning for Data-Efficient Deep Learning

Localization Distillation for Dense Object Detection

Unbiased Teacher for Semi-Supervised Object Detection

CoMatch: Semi-supervised Learning with Contrastive Graph Regularization

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Meta Pseudo Labels

DivideMix: Learning with Noisy Labels as Semi-supervised Learning

A Simple Framework for Contrastive Learning of Visual Representations

FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence

Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identification

ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring

Adversarial Transformations for Semi-Supervised Learning

Self-Training With Noisy Student Improves ImageNet Classification

Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning

MixMatch: A Holistic Approach to Semi-Supervised Learning

Unsupervised Data Augmentation for Consistency Training

DELTA: DEep Learning Transfer using Feature Map with Attention for Convolutional Networks

Learning Sound Event Classifiers from Web Audio with Noisy Labels

Scalable agent alignment via reward modeling: a research direction

AutoAugment: Learning Augmentation Policies from Data

Speech-Transformer: A No-Recurrence Sequence-to-Sequence Model for Speech Recognition

Strong Baselines for Neural Semi-Supervised Learning under Domain Shift

Explicit Inductive Bias for Transfer Learning with Convolutional Networks

EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification

Focal Loss for Dense Object Detection

Proximal Policy Optimization Algorithms

Adversarial Dropout for Supervised and Semi-supervised Learning

AgeDB: The First Manually Collected, In-the-Wild Age Database

Deep Reinforcement Learning from Human Preferences

Good Semi-supervised Learning That Requires a Bad GAN

Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning

Weight-averaged consistency targets improve semi-supervised deep learning results

Temporal Ensembling for Semi-Supervised Learning

SGDR: Stochastic Gradient Descent with Warm Restarts

Deep Expectation of Real and Apparent Age from a Single Image Without Facial Landmarks

Semi-Supervised Learning with Generative Adversarial Networks

Deep Residual Learning for Image Recognition

ESC: Dataset for Environmental Sound Classification

Character-level Convolutional Networks for Text Classification

Semi-supervised Learning with Ladder Networks

Adam: A Method for Stochastic Optimization

A Dataset and Taxonomy for Urban Sound Research

An Analysis of Single-Layer Networks in Unsupervised Feature Learning

ImageNet: A large-scale hierarchical image database

Semi-supervised learning by disagreement

Importance of Semantic Representation: Dataless Classification

Semi-Supervised Self-Training of Object Detection Models

Semi-supervised Learning by Entropy Minimization

Unsupervised Word Sense Disambiguation Rivaling Supervised Methods

RANK ANALYSIS OF INCOMPLETE BLOCK DESIGNS THE METHOD OF PAIRED COMPARISONS

Debiased Pseudo Labeling in Self-Training

SURF: Semi-supervised Reward Learning with Data Augmentation for Feedback-efficient Preference-based Reinforcement Learning

Co-Tuning for Transfer Learning

AutoMix: Mixup Networks for Sample Interpolation via Cooperative Barycenter Learning

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

2017) using a generator network and assigning it to a new ”generated