Evolving Knowledge Mining for Class Incremental Segmentation (2023-06-03T00:00:00.000000Z)

TL;DR

This paper investigates the efficient multi-grained knowledge reuse for CISS, and proposes a novel method, Evolving kNowleDge minING (ENDING), employing a frozen backbone, to demonstrate new state-of-the-art performance.

Abstract

Class Incremental Semantic Segmentation (CISS) has been a trend recently due to its great significance in real-world applications. Although the existing CISS methods demonstrate remarkable performance, they either leverage the high-level knowledge (feature) only while neglecting the rich and diverse knowledge in the low-level features, leading to poor old knowledge preservation and weak new knowledge exploration; or use multi-level features for knowledge distillation by retraining a heavy backbone, which is computationally intensive. In this paper, we for the first time investigate the efficient multi-grained knowledge reuse for CISS, and propose a novel method, Evolving kNowleDge minING (ENDING), employing a frozen backbone. ENDING incorporates two key modules: evolving fusion and semantic enhancement, for dynamic and comprehensive exploration of multi-grained knowledge. Evolving fusion is tailored to extract knowledge from individual low-level feature using a personalized lightweight network, which is generated from a meta-net, taking the high-level feature as input. This design enables the evolution of knowledge mining and fusing when applied to incremental new classes. In contrast, semantic enhancement is specifically crafted to aggregate prototype-based semantics from multi-level features, contributing to an enhanced representation. We evaluate our method on two widely used benchmarks and consistently demonstrate new state-of-the-art performance. The code is available at https://github.com/zhiheLu/ENDING_ISS.

Authors

Xinchao Wang

9 papers

Zhihe Lu

1 papers

Shuicheng Yan

1 papers

TL;DR

Abstract

Authors

References68 items

Endpoints Weight Fusion for Class Incremental Semantic Segmentation

Mining Unseen Classes via Regional Objectness: A Simple Baseline for Incremental Segmentation

ALIFE: Adaptive Logit Regularizer and Feature Replay for Incremental Semantic Segmentation

Decomposed Knowledge Distillation for Class-Incremental Semantic Segmentation

Representation Compensation Networks for Continual Semantic Segmentation

Masked-attention Mask Transformer for Universal Image Segmentation

Continual Learning on Noisy Data Streams via Self-Purified Replay

RECALL: Replay-based Continual Learning in Semantic Segmentation

Per-Pixel Classification is Not All You Need for Semantic Segmentation

SSUL: Semantic Segmentation with Unknown Label for Exemplar-based Class-Incremental Learning

OCNet: Object Context for Semantic Segmentation

Segmenter: Transformer for Semantic Segmentation

LIQA: Lifelong Blind Image Quality Assessment

On Learning the Geodesic Path for Incremental Learning

Rainbow Memory: Continual Learning with a Memory of Diverse Samples

Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations

Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers

PLOP: Learning without Forgetting for Continual Semantic Segmentation

Continual Prototype Evolution: Learning Online from Non-Stationary Data Streams

Topology-Preserving Class-Incremental Learning

Continual Deep Learning by Functional Regularisation of Memorable Past

PODNet: Pooled Outputs Distillation for Small-Tasks Incremental Learning

Semantic Drift Compensation for Class-Incremental Learning

Memory-Efficient Incremental Learning Through Feature Adaptation

Modeling the Background for Incremental Learning in Semantic Segmentation

Momentum Contrast for Unsupervised Visual Representation Learning

IL2M: Class Incremental Learning With Dual Memory

Using Hindsight to Anchor Past Knowledge in Continual Learning

Deep High-Resolution Representation Learning for Visual Recognition

Continual Learning by Asymmetric Loss Approximation With Single-Side Overestimation

Incremental Learning Techniques for Semantic Segmentation

Experience Replay for Continual Learning

CCNet: Criss-Cross Attention for Semantic Segmentation

Learning Without Memorizing

Dual Attention Network for Scene Segmentation

Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation

Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence

Overcoming catastrophic forgetting with hard attention to the task

Memory Aware Synapses: Learning what (not) to forget

Non-local Neural Networks

PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning

Scene Parsing through ADE20K Dataset

Rethinking Atrous Convolution for Semantic Image Segmentation

Gradient Episodic Memory for Continual Learning

Incremental Learning Through Deep Adaptation

Continual Learning with Deep Generative Replay

Encoder Based Lifelong Learning

Continual Learning Through Synaptic Intelligence

Pyramid Scene Parsing Network

Overcoming catastrophic forgetting in neural networks

iCaRL: Incremental Classifier and Representation Learning

Expert Gate: Lifelong Learning with a Network of Experts

Deeply Supervised Salient Object Detection with Short Connections

Learning without Forgetting

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

Deep Residual Learning for Image Recognition

Multi-Scale Context Aggregation by Dilated Convolutions

U-Net: Convolutional Networks for Biomedical Image Segmentation

Distilling the Knowledge in a Neural Network

Hypercolumns for object segmentation and fine-grained localization

Fully convolutional networks for semantic segmentation

Microsoft COCO: Common Objects in Context

ImageNet: A large-scale hierarchical image database

Learn++: an incremental learning algorithm for supervised neural networks

More Classifiers, Less Forgetting: A Generic Multi-classifier Paradigm for Incremental Learning

Incremental and Decremental Support Vector Machine Learning

Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem

International Journal of Computer Vision manuscript No. (will be inserted by the editor) The PASCAL Visual Object Classes (VOC) Challenge

Field of Study