Curriculum Learning: A Survey (2021-01-25T00:00:00.000000Z)

TL;DR

This survey shows how limits have been tackled in the literature, and presents different curriculum learning instantiations for various tasks in machine learning, and constructs a multi-perspective taxonomy of curriculum learning approaches by hand, considering various classification criteria.

Abstract

Training machine learning models in a meaningful order, from the easy samples to the hard ones, using curriculum learning can provide performance improvements over the standard training approach based on random data shuffling, without any additional computational costs. Curriculum learning strategies have been successfully employed in all areas of machine learning, in a wide range of tasks. However, the necessity of finding a way to rank the samples from easy to hard, as well as the right pacing function for introducing more difficult data can limit the usage of the curriculum approaches. In this survey, we show how these limits have been tackled in the literature, and we present different curriculum learning instantiations for various tasks in machine learning. We construct a multi-perspective taxonomy of curriculum learning approaches by hand, considering various classification criteria. We further build a hierarchical tree of curriculum learning methods using an agglomerative clustering algorithm, linking the discovered clusters with our taxonomy. At the end, we provide some interesting directions for future work.

Authors

Petru Soviany

1 papers

Radu Tudor Ionescu

13 papers

Paolo Rota

3 papers

TL;DR

Abstract

Authors

References212 items

Competence-based Multimodal Curriculum Learning for Medical Report Generation

FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling

A Novel Automated Curriculum Strategy to Solve Hard Sokoban Planning Instances

Eformer: Edge Enhancement based Transformer for Medical Image Denoising

Density-Based Dynamic Curriculum Learning for Intent Detection

UTNet: A Hybrid Transformer Architecture for Medical Image Segmentation

Enhancing Audio-Visual Association with Self-Supervised Curriculum Learning

Unsupervised MRI Reconstruction via Zero-Shot Learned Adversarial Transformers

Emerging Properties in Self-Supervised Vision Transformers

ViT-V-Net: Vision Transformer for Unsupervised Volumetric Medical Image Registration

A Survey on Curriculum Learning

CvT: Introducing Convolutions to Vision Transformers

Self-paced ensemble learning for speech and audio classification

UNETR: Transformers for 3D Medical Image Segmentation

Perceiver: General Perception with Iterative Attention

Meta-Curriculum Learning for Domain Adaptation in Neural Machine Translation

Reducing BERT Computation by Padding Removal and Curriculum Learning

Unsupervised Medical Image Alignment With Curriculum Learning

Automated curriculum learning for embodied agents a neuroevolutionary approach

TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation

Does the Order of Training Samples Matter? Improving Neural Data-to-Text Generation with Curriculum Learning

Curriculum Learning for Face Recognition

Transformers in Vision: A Survey

Automatic Curriculum Learning With Over-repetition Penalty for Dialogue Policy Learning

Automatic Curriculum Generation by Hierarchical Reinforcement Learning

Meta Automatic Curriculum Learning

Anomaly Detection in Video via Self-Supervised and Multi-Task Learning

A Comprehensive Survey on Curriculum Learning

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Deformable DETR: Deformable Transformers for End-to-End Object Detection

Guided Curriculum Learning for Walking Over Complex Terrain

A Curriculum Learning Based Approach to Captioning Ultrasound Images

Learn like a Pathologist: Curriculum Learning by Annotator Agreement for Histopathology Image Classification

Worst-Case-Aware Curriculum Learning for Zero and Few Shot Transfer

Curriculum Learning with Diversity for Supervised Computer Vision Tasks

Curriculum Learning with Hindsight Experience Replay for Sequential Object Manipulation Tasks

Accurate and Efficient Intracranial Hemorrhage Detection and Subtype Classification in 3D CT Scans with Convolutional and Long Short-Term Memory Neural Networks

Multi-Task Curriculum Framework for Open-Set Semi-Supervised Learning

Curriculum learning for multilevel budgeted combinatorial problems

Breaking the Curse of Space Explosion: Towards Efficient NAS with Curriculum Search

Curriculum Manager for Source Selection in Multi-Source Domain Adaptation

Uncertainty-Aware Curriculum Learning for Neural Machine Translation

Label Noise Robust Curriculum for Deep Paraphrase Identification

Curriculum Learning for Natural Language Understanding

PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning

Safe Reinforcement Learning via Curriculum Induction

Automatic Curriculum Learning through Value Disagreement

Dance Revolution: Long Sequence Dance Generation with Music via Curriculum Learning

Norm-Based Curriculum Learning for Neural Machine Translation

Language Models are Few-Shot Learners

End-to-End Object Detection with Transformers

On the Importance of Word and Sentence Representation Learning in Implicit Discourse Relation Classification

Curriculum Pre-training for End-to-End Speech Translation

Semi-Supervised Semantic Segmentation via Dynamic Self-Training and Class-Balanced Curriculum

Self-Induced Curriculum Learning in Self-Supervised Neural Machine Translation

Reinforced Curriculum Learning on Pre-trained Neural Machine Translation Models

Unsupervised feature selection by self-paced learning regularization

CurricularFace: Adaptive Curriculum Learning Loss for Deep Face Recognition

Curriculum DeepSDF

Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey

Curriculum By Smoothing

Accelerating Reinforcement Learning for Reaching Using Continuous Curriculum Learning

The Balanced Loss Curriculum Learning

Curriculum Audiovisual Learning

Curriculum Labeling: Self-paced Pseudo-Labeling for Semi-Supervised Learning

Parameter Continuation Methods for the Optimization of Deep Neural Networks

Fine-Tuning by Curriculum Learning for Non-Autoregressive Neural Machine Translation

Curriculum Self-Paced Learning for Cross-Domain Object Detection

Label-similarity Curriculum Learning

Self-Attention Enhanced CNNs and Collaborative Curriculum Learning for Distantly Supervised Relation Extraction

Expert-level detection of acute intracranial hemorrhage on head computed tomography using deep learning

Image Difficulty Curriculum for Generative Adversarial Networks (CuGAN)

Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environments

Medical-based Deep Curriculum Learning for Improved Fracture Classification

Self-Paced Contextual Reinforcement Learning

Rethinking Curriculum Learning with Incremental Labels and Adaptive Compensation