methodology-1

Class Incremental Learning

3260 papers • 126 benchmarks • 313 datasets

Incremental learning of a sequence of tasks when the task-ID is not available at test time.

(Image credit: Papersgraph)

Benchmarks

These leaderboards are used to track progress in class-incremental-learning-5

Trend

Dataset

Best Model

Actions

cifar100

CIFAR-100 - 50 classes + 10 steps of 5 classes

Libraries

i

Use these libraries to find class-incremental-learning-5 models and implementations

ContinualAI/avalanche

21 papers 1,654

Datasets

CIFAR-100

Subtasks

Few-Shot Class-Incremental Learning Class-Incremental Semantic Segmentation Non-exemplar-based Class Incremental Learning

Most implemented papers

Overcoming catastrophic forgetting in neural networks

D. Hassabis, Razvan Pascanu, J. Kirkpatrick, Andrei A. Rusu, J. Veness, D. Kumaran, C. Clopath, R. Hadsell, John Quan, Guillaume Desjardins, Neil C. Rabinowitz, Kieran Milan, Tiago Ramalho, A. Grabska-Barwinska•Thu Dec 01 2016

It is shown that it is possible to overcome the limitation of connectionist models and train networks that can maintain expertise on tasks that they have not experienced for a long time and selectively slowing down learning on the weights important for previous tasks.

Content

CIFAR-100 - 50 classes + 5 steps of 10 classes

Cifar100-B0(10 tasks)-no-exemplars

CIFAR100-B0(50 tasks)-no-exemplars

Cifar100-B0(20 tasks)-no-exemplars

g-u-n/pycil

16 papers 679

mmasana/FACIL

9 papers 494

aimagelab/mammoth

2 papers 456

8621

0

Paper Graph

Supervised Contrastive Learning

Phillip Isola, Yonglong Tian, Aaron Sarna, Ce Liu, Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Maschinot, Dilip Krishnan•Wed Apr 22 2020

A novel training methodology that consistently outperforms cross entropy on supervised learning tasks across different architectures and data augmentations is proposed, and the batch contrastive loss is modified, which has recently been shown to be very effective at learning powerful representations in the self-supervised setting.

5667 0

Paper Graph

iCaRL: Incremental Classifier and Representation Learning

Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, G. Sperl, Christoph H. Lampert•Tue Nov 22 2016

iCaRL can learn many classes incrementally over a long period of time where other strategies quickly fail, and distinguishes it from earlier works that were fundamentally limited to fixed data representations and therefore incompatible with deep learning architectures.

4270 0

Paper Graph

Learning without Forgetting

Derek Hoiem, Zhizhong Li•Tue Jun 28 2016

This work proposes the Learning without Forgetting method, which uses only new task data to train the network while preserving the original capabilities, and performs favorably compared to commonly used feature extraction and fine-tuning adaption techniques.

5196 0

Paper Graph

Continual Learning with Deep Generative Replay

Jiwon Kim, Jung Kwon Lee, Hanul Shin, Jaehong Kim•Sun Apr 30 2017

The Deep Generative Replay is proposed, a novel framework with a cooperative dual model architecture consisting of a deep generative model ("generator") and a task solving model ("solver"), with only these two models, training data for previous tasks can easily be sampled and interleaved with those for a new task.

2299 0

Paper Graph

Three scenarios for continual learning

A. Tolias, Gido M. van de Ven•Sun Apr 14 2019

Three continual learning scenarios are described based on whether at test time task identity is provided and--in case it is not--whether it must be inferred, and it is found that regularization-based approaches fail and that replaying representations of previous experiences seems required for solving this scenario.

1008 0

Paper Graph

On Tiny Episodic Memories in Continual Learning

Marc'Aurelio Ranzato, Philip H. S. Torr, Marcus Rohrbach, Arslan Chaudhry, P. Dokania, Mohamed Elhoseiny, Thalaiyasingam Ajanthan•Tue Feb 26 2019

This work empirically analyze the effectiveness of a very small episodic memory in a CL setup where each training example is only seen once and finds that repetitive training on even tiny memories of past tasks does not harm generalization, on the contrary, it improves it.

789 0

Paper Graph

Rehearsal-Free Continual Learning over Small Non-I.I.D. Batches

Vincenzo Lomonaco, D. Maltoni, Lorenzo Pellegrini•Sun Jul 07 2019

A novel continual learning protocol based on the CORe50 benchmark is introduced and two rehearsal-free continual learning techniques are proposed, CWR* and AR1*, that can learn effectively even in the challenging case of nearly 400 small non-i.i.d. incremental batches.

71 0

Paper Graph

A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks

Jinwoo Shin, Honglak Lee, Kimin Lee, Kibok Lee•Mon Jul 09 2018

This paper proposes a simple yet effective method for detecting any abnormal samples, which is applicable to any pre-trained softmax neural classifier, and obtains the class conditional Gaussian distributions with respect to (low- and upper-level) features of the deep models under Gaussian discriminant analysis.

2403 0

Paper Graph

Gradient based sample selection for online continual learning

Yoshua Bengio, Min Lin, Rahaf Aljundi, Baptiste Goujaud•Tue Mar 19 2019

This work formulation of sample selection as a constraint reduction problem based on the constrained optimization view of continual learning shows that it is equivalent to maximizing the diversity of samples in the replay buffer with parameters gradient as the feature.

955 0

Paper Graph

Adding a benchmark result helps the community track progress.