PiCO+: Contrastive Label Disambiguation for Robust Partial Label Learning (2022-01-22T00:00:00.000000Z)

TL;DR

This work develops a novel label disambiguation algorithm PiCO that consists of a contrastive learning module along with a novel class prototype-based disambiguation method that can be rigorously justified from an expectation-maximization (EM) algorithm perspective.

Abstract

Partial label learning (PLL) is an important problem that allows each training example to be labeled with a coarse candidate set with the ground-truth label included. However, in a more practical but challenging scenario, the annotator may miss the ground-truth and provide a wrong candidate set, which is known as the noisy PLL problem. To remedy this problem, we propose the PiCO+ framework that simultaneously disambiguates the candidate sets and mitigates label noise. Core to PiCO+, we develop a novel label disambiguation algorithm PiCO that consists of a contrastive learning module along with a novel class prototype-based disambiguation method. Theoretically, we show that these two components are mutually beneficial, and can be rigorously justified from an expectation-maximization (EM) algorithm perspective. To handle label noise, we extend PiCO to PiCO+, which further performs distance-based clean sample selection, and learns robust classifiers by a semi-supervised contrastive learning algorithm. Beyond this, we further investigate the robustness of PiCO+ in the context of out-of-distribution noise and incorporate a novel energy-based rejection method for improved robustness. Extensive experiments demonstrate that our proposed methods significantly outperform the current state-of-the-art approaches in standard and noisy PLL tasks and even achieve comparable results to fully supervised learning.

Authors

J. Zhao

4 papers

Gang Niu

11 papers

Yixuan Li

9 papers

TL;DR

Abstract

Authors

References77 items

Energy-based Out-of-Distribution Detection for Graph Neural Networks

Learning from Multiple Annotator Noisy Labels via Sample-wise Label Fusion

Out-of-distribution Detection with Deep Nearest Neighbors

Generalized Out-of-Distribution Detection: A Survey

NGC: A Unified Framework for Learning with Open-World Noisy Data

On the Robustness of Average Losses for Partial-Label Learning

Leveraged Weighted Loss for Partial Label Learning

Sample Selection with Uncertainty of Losses for Learning with Noisy Labels

Semi-Supervised Contrastive Learning With Similarity Co-Calibration

Open-World Semi-Supervised Learning

CoMatch: Semi-supervised Learning with Contrastive Graph Regularization

A Survey of Label-noise Representation Learning: Past, Present and Future

Unsupervised Semantic Aggregation and Deformable Template Matching for Semi-Supervised Learning

MoPro: Webly Supervised Learning with Momentum Prototypes

Attribute Prototype Network for Zero-Shot Learning

Provably Consistent Partial-Label Learning

Human-Driven FOL Explanations of Deep Learning

Unsupervised Learning of Visual Features by Contrasting Cluster Assignments

Dual T: Reducing Estimation Error for Transition Matrix in Label-noise Learning

Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere

Prototypical Contrastive Learning of Unsupervised Representations

Supervised Contrastive Learning

Combating Noisy Labels by Agreement: A Joint Training Method with Co-Regularization

Progressive Identification of True Labels for Partial-Label Learning

DivideMix: Learning with Noisy Labels as Semi-supervised Learning

A Simple Framework for Contrastive Learning of Visual Representations

FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence

Momentum Contrast for Unsupervised Visual Representation Learning

SELF: Learning to Filter Noisy Labels with Self-Ensembling

RandAugment: Practical data augmentation with no separate search

Symmetric Cross Entropy for Robust Learning With Noisy Labels

Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning

Adaptive Graph Guided Disambiguation for Partial Label Learning

Partial Label Learning via Label Enhancement

MixMatch: A Holistic Approach to Semi-Supervised Learning

A Theoretical Analysis of Contrastive Unsupervised Representation Learning

Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting

Attention-Based Prototypical Learning Towards Interpretable, Confident and Robust Deep Neural Networks

GM-PLL: Graph Matching Based Partial Label Learning

Representation Learning with Contrastive Predictive Coding

Places: A 10 Million Image Database for Scene Recognition

Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels

Co-teaching: Robust training of deep neural networks with extremely noisy labels

Learning to Reweight Examples for Robust Deep Learning

MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels

Prototypical Networks for Few-shot Learning

Learning from Ambiguously Labeled Face Images

Understanding deep learning requires rethinking generalization

A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks

Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach

Partial Label Learning via Feature-Aware Disambiguation

Solving the Partial Label Learning Problem: An Instance-Based Approach

Learning with Noisy Labels

A Conditional Multinomial Mixture Model for Superset Label Learning

Learning from Partial Labels

Learning from Candidate Labeling Sets

Caltech-UCSD Birds 200

Automated Flower Classification over a Large Number of Classes

Classification with partial labels

Clustering on the Unit Hypersphere using von Mises-Fisher Distributions

Learning from ambiguously labeled examples

Article

PiCO: Contrastive Label Disambiguation for Partial Label Learning

Online Partial Label Learning

Learning with Multiple Complementary Labels

“Enhancing the reliability of out-of-distributionimagedetectioninneuralnetworks,”in

“De-scribing textures in the wild,”

Reading Digits in Natural Images with Unsupervised Feature Learning

Learning Multiple Layers of Features from Tiny Images

Visualizing Data using t-SNE

Learning with Multiple Labels

Convergence Properties of the K-Means Algorithms

Additionally

committee member for IJCAI 2021 and AAAI 2022, member (or reviewer) for other top conferences

Empirically, our proposed PiCO+ framework establishes the state-of-the-art performance on various PLL tasks, showing its robustness to label ambiguity, label noise, and out-of-distribution data

learning, trustworthy deep learning, and data science