Improving Replay-Based Continual Semantic Segmentation with Smart Data Selection (2022-09-20T00:00:00.000000Z)

TL;DR

The findings suggest that in a class-incremental setting, it is critical to achieve a uniform distribution for the different classes in the buffer to avoid a bias towards newly learned classes, and that the effective sampling methods help to decrease the representation shift significantly in early layers, which is a major cause of forgetting in domain- increasemental learning.

Abstract

Abhstract- Continual learning for Semantic Segmentation (CSS) is a rapidly emerging field, in which the capabilities of the segmentation model are incrementally improved by learning new classes or new domains. A central challenge in Continual Learning is overcoming the effects of catastrophic forgetting, which refers to the sudden drop in accuracy on previously learned tasks after the model is trained on new classes or domains. In continual classification this challenge is often overcome by replaying a small selection of samples from previous tasks, however replay is rarely considered in CSS. Therefore, we investigate the influences of various replay strategies for semantic segmentation and evaluate them in class- and domain-incremental settings. Our findings suggest that in a class-incremental setting, it is critical to achieve a uniform distribution for the different classes in the buffer to avoid a bias towards newly learned classes. In the domainincremental setting, it is most effective to select buffer samples by uniformly sampling from the distribution of learned feature representations or by choosing samples with median entropy. Finally, we observe that the effective sampling methods help to decrease the representation shift significantly in early layers, which is a major cause of forgetting in domain-incremental learning.

Authors

Tobias Kalb

2 papers

J. Beyerer

2 papers

Björn Mauthe

1 papers

TL;DR

Abstract

Authors

References48 items

Wanderlust: Online Continual Object Detection in the Real World

RECALL: Replay-based Continual Learning in Semantic Segmentation

Continual Learning for Class- and Domain-Incremental Semantic Segmentation

Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentation

Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations

Towards Open World Object Detection

PLOP: Learning without Forgetting for Continual Semantic Segmentation

A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks

Class-Incremental Learning: Survey and Performance Evaluation on Image Classification

Remembering for the Right Reasons: Explanations Reduce Catastrophic Forgetting

RODEO: Replay for Online Object Detection

Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics

Class-Incremental Learning for Semantic Segmentation Re-Using Neither Old Data Nor Old Labels

Modeling the Background for Incremental Learning in Semantic Segmentation

REMIND Your Neural Network to Prevent Catastrophic Forgetting

A Continual Learning Survey: Defying Forgetting in Classification Tasks

Incremental Learning Techniques for Semantic Segmentation

Lifelong GAN: Continual Learning for Conditional Image Generation

Large Scale Incremental Learning

Similarity of Neural Network Representations Revisited

Gradient based sample selection for online continual learning

Superposition of many models into one

Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines

Incremental Learning for Semantic Segmentation of Large-Scale Remote Sensing Data

Efficient Lifelong Learning with A-GEM

Memory Replay GANs: learning to generate images from new categories without forgetting

End-to-End Incremental Learning

BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning

UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction

Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence

The Unreasonable Effectiveness of Deep Features as a Perceptual Metric

Memory Aware Synapses: Learning what (not) to forget

Gradient Episodic Memory for Continual Learning

Continual Learning with Deep Generative Replay

Continual Learning Through Synaptic Intelligence

Overcoming catastrophic forgetting in neural networks

iCaRL: Incremental Classifier and Representation Learning

Expert Gate: Lifelong Learning with a Network of Experts

Learning without Forgetting

The Cityscapes Dataset for Semantic Urban Scene Understanding

Distilling the Knowledge in a Neural Network

Blind/Referenceless Image Spatial Quality Evaluator

Herding dynamical weights to learn

Nonlinear total variation based noise removal algorithms

ERFNet: Efficient Residual Factorized ConvNet for Real-Time Semantic Segmentation

A Mathematical Theory of Communication

Catastrophic Forgetting in Connectionist Networks: Causes, Consequences and Solutions

Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem

Field of Study

Journal Information

Name

Page