Attribution-aware Weight Transfer: A Warm-Start Initialization for Class-Incremental Semantic Segmentation (2022-10-13T00:00:00.000000Z)

TL;DR

This work proposes to address the background shift with a novel classifier initialization method which employs gradient-based attribution to identify the most relevant weights for new classes from the classifier’s weights for the previous background and transfers these weights to the new classifier.

Abstract

In class-incremental semantic segmentation (CISS), deep learning architectures suffer from the critical problems of catastrophic forgetting and semantic background shift. Although recent works focused on these issues, existing classifier initialization methods do not address the background shift problem and assign the same initialization weights to both background and new foreground class classifiers. We propose to address the background shift with a novel classifier initialization method which employs gradient-based attribution to identify the most relevant weights for new classes from the classifier’s weights for the previous background and transfers these weights to the new classifier. This warm-start weight initialization provides a general solution applicable to several CISS methods. Furthermore, it accelerates learning of new classes while mitigating forgetting. Our experiments demonstrate significant improvement in mIoU compared to the state-of-the-art CISS methods on the Pascal-VOC 2012, ADE20K and Cityscapes datasets.

Authors

Joost van de Weijer

11 papers

Dipam Goswami

2 papers

D. Stricker

6 papers

TL;DR

Abstract

Authors

References58 items

Uncertainty-aware Contrastive Distillation for Incremental Semantic Segmentation

Representation Compensation Networks for Continual Semantic Segmentation

RECALL: Replay-based Continual Learning in Semantic Segmentation

Continual Learning for Class- and Domain-Incremental Semantic Segmentation

Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentation

SSUL: Semantic Segmentation with Unknown Label for Exemplar-based Class-Incremental Learning

Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations

Self-Training for Class-Incremental Semantic Segmentation

PLOP: Learning without Forgetting for Continual Semantic Segmentation

Class-Incremental Learning: Survey and Performance Evaluation on Image Classification

Captum: A unified and generic model interpretability library for PyTorch

Class-Incremental Learning for Semantic Segmentation Re-Using Neither Old Data Nor Old Labels

PODNet: Pooled Outputs Distillation for Small-Tasks Incremental Learning

Semantic Drift Compensation for Class-Incremental Learning

Strip Pooling: Rethinking Spatial Pooling for Scene Parsing

Modeling the Background for Incremental Learning in Semantic Segmentation

ScaIL: Classifier Weights Scaling for Class Incremental Learning

IL2M: Class Incremental Learning With Dual Memory

A Continual Learning Survey: Defying Forgetting in Classification Tasks

Incremental Learning Techniques for Semantic Segmentation

Learning a Unified Classifier Incrementally via Rebalancing

Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting

Learning Without Memorizing

Extending pretrained segmentation networks with additional anatomical structures

Incremental Learning for Semantic Segmentation of Large-Scale Remote Sensing Data

Efficient Lifelong Learning with A-GEM

PSANet: Point-wise Spatial Attention Network for Scene Parsing

End-to-End Incremental Learning

Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation

Incremental Classifier Learning with Generative Adversarial Networks

Low-Shot Learning with Imprinted Weights

In-place Activated BatchNorm for Memory-Optimized Training of DNNs

FearNet: Brain-Inspired Model for Incremental Learning

Memory Aware Synapses: Learning what (not) to forget

Incremental Learning of Object Detectors without Catastrophic Forgetting

Lifelong Learning with Dynamically Expandable Networks

Scene Parsing through ADE20K Dataset

Rethinking Atrous Convolution for Semantic Image Segmentation

Gradient Episodic Memory for Continual Learning

Continual Learning with Deep Generative Replay

Learning Important Features Through Propagating Activation Differences

Axiomatic Attribution for Deep Networks

Pyramid Scene Parsing Network

Overcoming catastrophic forgetting in neural networks

iCaRL: Incremental Classifier and Representation Learning

RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation

Learning without Forgetting

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

The Cityscapes Dataset for Semantic Urban Scene Understanding

Deep Residual Learning for Image Recognition

Attention to Scale: Scale-Aware Semantic Image Segmentation

On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation

Fully convolutional networks for semantic segmentation

The Pascal Visual Object Classes Challenge: A Retrospective

Visualizing and Understanding Convolutional Networks

ImageNet: A large-scale hierarchical image database

SSUL+AWT (f) GT Figure 8: Visualization of predictions using MiB, MiB+AWT, SSUL and SSUL+AWT in 10-1 setting on test images of Pascal-VOC

Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names