iCaRL: Incremental Classifier and Representation Learning (2016-11-23T00:00:00.000000Z)

TL;DR

iCaRL can learn many classes incrementally over a long period of time where other strategies quickly fail, and distinguishes it from earlier works that were fundamentally limited to fixed data representations and therefore incompatible with deep learning architectures.

Abstract

A major open problem on the road to artificial intelligence is the development of incrementally learning systems that learn about more and more concepts over time from a stream of data. In this work, we introduce a new training strategy, iCaRL, that allows learning in such a class-incremental way: only the training data for a small number of classes has to be present at the same time and new classes can be added progressively. iCaRL learns strong classifiers and a data representation simultaneously. This distinguishes it from earlier works that were fundamentally limited to fixed data representations and therefore incompatible with deep learning architectures. We show by experiments on CIFAR-100 and ImageNet ILSVRC 2012 data that iCaRL can learn many classes incrementally over a long period of time where other strategies quickly fail.

Authors

Sylvestre-Alvise Rebuffi

3 papers

Alexander Kolesnikov

13 papers

G. Sperl

1 papers

TL;DR

Abstract

Authors

References45 items

Encoder Based Lifelong Learning

Overcoming catastrophic forgetting in neural networks

Learning without Forgetting

Progressive Neural Networks

Convolutional Neural Fabrics

Cross-Stitch Networks for Multi-task Learning

Incremental Learning of Random Forests for Large-Scale Image Classification

Deep Residual Learning for Image Recognition

Classifier adaptation at prediction time

Distilling the Knowledge in a Neural Network

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Adam: A Method for Stochastic Optimization

Towards Open World Recognition

Curriculum learning of multiple tasks

Error-Driven Incremental Learning in Deep Convolutional Neural Network for Large-Scale Image Classification

ImageNet Large Scale Visual Recognition Challenge

Incremental Learning of NCM Forests for Large-Scale Image Classification

Enriching Visual Knowledge Bases via Object Discovery and Segmentation

Learning Everything about Anything: Webly-Supervised Visual Concept Learning

Probability Models for Open Set Recognition

Data-driven exemplar model selection

An Empirical Investigation of Catastrophic Forgeting in Gradient-Based Neural Networks

NEIL: Extracting Visual Knowledge from Web Data

Distance-Based Image Classification: Generalizing to New Classes at Near-Zero Cost

Toward Open Set Recognition

From N to N+1: Multiclass Transfer Incremental Learning

Metric Learning for Large Scale Image Classification: Generalizing to New Classes at Near-Zero Cost

Representation Learning: A Review and New Perspectives

Sparse Subspace Clustering: Algorithm, Theory, and Applications

Herding dynamical weights to learn

Open set face recognition using transduction

Learn++: an incremental learning algorithm for supervised neural networks

Catastrophic forgetting in connectionist networks

Avoiding catastrophic forgetting by coupling two reverberating neural networks

Catastrophic Forgetting, Rehearsal and Pseudorehearsal

Catastrophic Interference in Connectionist Networks: Can It Be Predicted, Can It Be Prevented?

Dropout: a simple way to prevent neural networks from overfitting

Learning Multiple Layers of Features from Tiny Images

Learn$^{++}$ .NC: Combining Ensemble of Classifiers With Dynamically Weighted Consult-and-Vote for Efficient Incremental Learning of New Classes

Catastophic Forgetting in Neural Networks

Avoiding catastrophic forgetting by coupling two reverberating neural networks

GradientBased Learning Applied to Document Recognition

Gradient-based learning applied to document recognition

Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem

Attribute-Based Classification for Zero-Shot Visual Object Categorization

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names