Boosting Active Learning via Improving Test Performance (2021-12-10T00:00:00.000000Z)

TL;DR

This work theoretically proves that selecting unlabeled data of higher gradient norm leads to a lower upper-bound of test loss, resulting in a better test performance, and proposes two schemes, namely expected-grad norm and entropy-gradnorm, which are integrated in a universal AL framework.

Abstract

Central to active learning (AL) is what data should be selected for annotation. Existing works attempt to select highly uncertain or informative data for annotation. Nevertheless, it remains unclear how selected data impacts the test performance of the task model used in AL. In this work, we explore such an impact by theoretically proving that selecting unlabeled data of higher gradient norm leads to a lower upper-bound of test loss, resulting in a better test performance. However, due to the lack of label information, directly computing gradient norm for unlabeled data is infeasible. To address this challenge, we propose two schemes, namely expected-gradnorm and entropy-gradnorm. The former computes the gradient norm by constructing an expected empirical loss while the latter constructs an unsupervised loss with entropy. Furthermore, we integrate the two schemes in a universal AL framework. We evaluate our method on classical image classification and semantic segmentation tasks. To demonstrate its competency in domain applications and its robustness to noise, we also validate our method on a cellular imaging analysis task, namely cryo-Electron Tomography subtomogram classification. Results demonstrate that our method achieves superior performance against the state of the art. We refer readers to https://arxiv.org/pdf/2112.05683.pdf for the full version of this paper which includes the appendix and source code link.

Authors

Guosheng Hu

3 papers

Tianyang Wang

1 papers

Xingjian Li

1 papers

TL;DR

Abstract

Authors

References54 items

Influence Selection for Active Learning

Sequential Graph Convolutional Network for Active Learning

State-Relabeling Adversarial Active Learning

A General Framework for Uncertainty Estimation in Deep Learning

Deep Active Learning with Adaptive Acquisition

Learning Loss for Active Learning

MixMatch: A Holistic Approach to Semi-Supervised Learning

Variational Adversarial Active Learning

Cost-Sensitive Active Learning for Intracranial Hemorrhage Detection

Adversarial Sampling for Active Learning

Efficient Active Learning for Image Classification and Segmentation using a Sample Selection and Conditional Generative Adversarial Network

The Power of Ensembles for Active Learning in Image Classification

Knowledge Transfer with Jacobian Matching

Adversarial Active Learning for Deep Networks: a Margin Based Approach

Stronger generalization bounds for deep nets via a compression approach

Visualizing the Loss Landscape of Neural Nets

Cost-Effective Active Learning for Melanoma Segmentation

Active Learning for Classification with Maximum Model Change

Active Learning for Convolutional Neural Networks: A Core-Set Approach

Dilated Residual Networks

Understanding Black-box Predictions via Influence Functions

Learning Active Learning from Data

Deep Bayesian Active Learning with Image Data

Generative Adversarial Active Learning

Convolutional Neural Networks for Automated Annotation of Cellular Cryo-Electron Tomograms

Semi-Supervised Classification with Graph Convolutional Networks

Piecewise convexity of artificial neural networks

The Cityscapes Dataset for Semantic Urban Scene Understanding

Deep Residual Learning for Image Recognition

Context Aware Active Learning of Activity Recognition Models

Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning

Multi-Class Active Learning by Uncertainty Sampling with Diversity Maximization

Adam: A Method for Stochastic Optimization

Very Deep Convolutional Networks for Large-Scale Image Recognition

Hierarchical Subquery Evaluation for Active Learning on a Graph

Maximizing Expected Model Change for Active Learning in Regression

A Convex Optimization Framework for Active Learning

ImageNet: A large-scale hierarchical image database

Multiple-Instance Active Learning

One-shot learning of object categories

Semi-supervised Learning by Entropy Minimization

Active learning using pre-clustering

AUTO-ENCODING VARIATIONAL BAYES

SHREC’19 Track: Classification in Cryo-Electron Tomograms

GENERATIVE ADVERSARIAL NETS

Assuming that f is locally μ-strongly convex and L-smooth at eachw, we have μI H = ∇2f(w

2017), which is an important domain task in computational biology. This experiment also demonstrates the robustness of our method to noise since part of the used data contains high level of noise

2016). As shown in Table 6, ResNet-50 achieves the best test performance when it is trained on the data

CH×W possibilities when making label assumptions, where C denotes the number of classes for each pixel

Reading Digits in Natural Images with Unsupervised Feature Learning

Large-Scale Machine Learning with Stochastic Gradient Descent

Learning Multiple Layers of Features from Tiny Images

Link-based Active Learning

Toward Optimal Active Learning through Monte Carlo Estimation of Error Reduction

Field of Study

Venue Information

Name

Type

URL

Alternate Names