NAPA-VQ: Neighborhood Aware Prototype Augmentation with Vector Quantization for Continual Learning (2023-08-18T00:00:00.000000Z)

TL;DR

NAPA-VQ: Neighborhood Aware Prototype Augmentation with Vector Quantization, a framework that reduces class overlap in NECIL, and outperforms the State-of-the-art NECIL methods by an average improvement of 5%, 2%, and 4% in accuracy and 10%, 3%, and 9% in forgetting respectively.

Abstract

Catastrophic forgetting; the loss of old knowledge upon acquiring new knowledge, is a pitfall faced by deep neural networks in real-world applications. Many prevailing solutions to this problem rely on storing exemplars (previously encountered data), which may not be feasible in applications with memory limitations or privacy constraints. Therefore, the recent focus has been on Non-Exemplar based Class Incremental Learning (NECIL) where a model incrementally learns about new classes without using any past exemplars. However, due to the lack of old data, NECIL methods struggle to discriminate between old and new classes causing their feature representations to overlap. We propose NAPA-VQ: Neighborhood Aware Prototype Augmentation with Vector Quantization, a framework that reduces this class overlap in NECIL. We draw inspiration from Neural Gas to learn the topological relationships in the feature space, identifying the neighboring classes that are most likely to get confused with each other. This neighborhood information is utilized to enforce strong separation between the neighboring classes as well as to generate old class representative prototypes that can better aid in obtaining a discriminative decision boundary between old and new classes. Our comprehensive experiments on CIFAR-100, TinyImageNet, and ImageNet-Subset demonstrate that NAPA-VQ outperforms the State-of-the-art NECIL methods by an average improvement of 5%, 2%, and 4% in accuracy and 10%, 3%, and 9% in forgetting respectively. Our code can be found in https://github.com/TamashaM/NAPA-VQ.git.

Authors

T. Malepathirana

1 papers

Damith A. Senanayake

1 papers

Saman K. Halgamuge

1 papers

TL;DR

Abstract

Authors

References61 items

A Closer Look at Rehearsal-Free Continual Learning *

Self-Sustaining Representation Expansion for Non-Exemplar Class-Incremental Learning

Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning

Prototype Augmentation and Self-Supervision for Incremental Learning

Embracing Change: Continual Learning in Deep Neural Networks

Feature Space Augmentation for Long-Tailed Data

Catastrophic forgetting and mode collapse in GANs

Few-Shot Class-Incremental Learning

Semantic Drift Compensation for Class-Incremental Learning

A Continual Learning Survey: Defying Forgetting in Classification Tasks

Learning a Unified Classifier Incrementally via Rebalancing

Gradient based sample selection for online continual learning

Class-incremental Learning via Deep Model Consolidation

On Tiny Episodic Memories in Continual Learning

Learning Without Memorizing

Efficient Lifelong Learning with A-GEM

Generative replay with feedback connections as a general strategy for continual learning

End-to-End Incremental Learning

Robust Classification with Convolutional Prototype Learning

Rotate your Networks: Better Weight Consolidation and Less Catastrophic Forgetting

Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence

Overcoming catastrophic forgetting with hard attention to the task

Memory Aware Synapses: Learning what (not) to forget

PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning

Few-shot learning in deep networks through global prototyping

Fusion of deep learning architectures, multilayer feedforward networks and learning vector quantizers for deep classification learning

Lifelong Learning with Dynamically Expandable Networks

Gradient Episodic Memory for Continual Learning

Continual Learning in Generative Adversarial Nets

Continual Learning with Deep Generative Replay

Continual Learning Through Synaptic Intelligence

PathNet: Evolution Channels Gradient Descent in Super Neural Networks

Overcoming catastrophic forgetting in neural networks

iCaRL: Incremental Classifier and Representation Learning

Expert Gate: Lifelong Learning with a Network of Experts

Learning without Forgetting

Progressive Neural Networks

Deep Residual Learning for Image Recognition

Fast R-CNN

Fully convolutional networks for semantic segmentation

ImageNet classification with deep convolutional neural networks

ImageNet: A large-scale hierarchical image database

Supervised Neural Gas with General Similarity Measure

Generalized Learning Vector Quantization

The self-organizing map

Improved versions of learning vector quantization

Discriminatory Analysis - Nonparametric Discrimination: Consistency Properties

Learning

Class-Incremental Learning via Dual Augmentation

Incremental few-shot learning via vector quantization in deep embedded space

More Classifiers, Less Forgetting: A Generic Multi-classifier Paradigm for Incremental Learning

Topology-Preserving Class-Incremental Learning for Image Classificatio

Advances in Neural Information Processing Systems , volume 30

Deep Learning Vector Quantization

Spartan performance and flexibility: An hpc-cloud chimera

Tiny ImageNet Visual Recognition Challenge

Learning Multiple Layers of Features from Tiny Images

Visualizing Data using t-SNE

Stochastic Neighbor Embedding

A Growing Neural Gas Network Learns Topologies

applicable license agreement with IEEE. Restrictions apply

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names