Neuromorphic Data Augmentation for Training Spiking Neural Networks (2022-03-11T00:00:00.000000Z)

TL;DR

This work proposes Neuromorphic Data Augmentation (NDA), a family of geometric augmentations specifically designed for event-based datasets with the goal of significantly stabilizing the SNN training and reducing the generalization gap between training and test performance.

Abstract

Developing neuromorphic intelligence on event-based datasets with Spiking Neural Networks (SNNs) has recently attracted much research attention. However, the limited size of event-based datasets makes SNNs prone to overfitting and unstable convergence. This issue remains unexplored by previous academic works. In an effort to minimize this generalization gap, we propose Neuromorphic Data Augmentation (NDA), a family of geometric augmentations specifically designed for event-based datasets with the goal of significantly stabilizing the SNN training and reducing the generalization gap between training and test performance. The proposed method is simple and compatible with existing SNN training pipelines. Using the proposed augmentation, for the first time, we demonstrate the feasibility of unsupervised contrastive learning for SNNs. We conduct comprehensive experiments on prevailing neuromorphic vision benchmarks and show that NDA yields substantial improvements over previous state-of-the-art results. For example, the NDA-based SNN achieves accuracy gain on CIFAR10-DVS and N-Caltech 101 by 10.1% and 13.7%, respectively. Code is available on GitHub https://github.com/Intelligent-Computing-Lab-Yale/NDA_SNN

Authors

Yuhang Li

2 papers

Youngeun Kim

2 papers

Hyoungseob Park

1 papers

TL;DR

Abstract

Authors

References75 items

RecDis-SNN: Rectifying Membrane Potential Distribution for Directly Training Spiking Neural Networks

Converting Artificial Neural Networks to Spiking Neural Networks via Parameter Calibration

Temporal Efficient Training of Spiking Neural Network via Gradient Re-weighting

ES-ImageNet: A Million Event-Stream Classification Dataset for Spiking Neural Networks

Optimizing Deeper Spiking Neural Networks for Dynamic Vision Sensing

CarSNN: An Efficient Spiking Neural Network for Event-Based Autonomous Cars on the Loihi Neuromorphic Research Processor

A Free Lunch From ANN: Towards Efficient, Accurate Spiking Neural Networks Calibration

EventDrop: data augmentation for event-based learning

2022 roadmap on neuromorphic computing and engineering

Diversifying Sample Generation for Accurate Data-Free Quantization

Optimal Conversion of Conventional Artificial Neural Networks to Spiking Neural Networks

Exploring Simple Siamese Representation Learning

MixMix: All You Need for Data-Free Compression Are Feature and Data Mixing

LIAF-Net: Leaky Integrate and Analog Fire Network for Lightweight and Efficient Spatiotemporal Information Processing

Going Deeper With Directly-Trained Larger Spiking Neural Networks

Revisiting Batch Normalization for Training Low-Latency Deep Spiking Neural Networks From Scratch

Deep Spiking Neural Network: Energy Efficiency Through Time Based Coding

DIET-SNN: Direct Input Encoding With Leakage and Threshold Optimization in Deep Spiking Neural Networks

Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks

Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning

Efficient Processing of Spatio-Temporal Data Streams With Spiking Neural Networks

Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagation

Improved Baselines with Momentum Contrastive Learning

Effective AER Object Classification Using Segmented Probability-Maximization Learning in Spiking Neural Networks

A Simple Framework for Contrastive Learning of Visual Representations

PyHessian: Neural Networks Through the Lens of the Hessian

Momentum Contrast for Unsupervised Visual Representation Learning

Towards spike-based machine intelligence with neuromorphic computing

Randaugment: Practical automated data augmentation with a reduced search space

A survey on Image Data Augmentation for Deep Learning

Learning Data Augmentation Strategies for Object Detection

CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features

Fast AutoAugment

Event-Based Vision: A Survey

Large Scale GAN Training for High Fidelity Natural Image Synthesis

Direct Training for Spiking Neural Networks: Faster, Larger, Better

SLAYER: Spike Layer Error Reassignment in Time

BindsNET: A Machine Learning-Oriented Spiking Neural Networks Library in Python

AutoAugment: Learning Augmentation Policies from Data

HATS: Histograms of Averaged Time Surfaces for Robust Event-Based Object Classification

Loihi: A Neuromorphic Manycore Processor with On-Chip Learning

DART: Distribution Aware Retinal Transform for Event-Based Cameras

Progressive Growing of GANs for Improved Quality, Stability, and Variation

mixup: Beyond Empirical Risk Minimization

Deep fully convolutional networks with random data augmentation for enhanced generalization in road detection

Random Erasing Data Augmentation

Improved Regularization of Convolutional Neural Networks with Cutout

Wasserstein Generative Adversarial Networks

Large Scale Labelled Video Data Augmentation for Semantic Segmentation in Driving Scenarios

HOTS: A Hierarchy of Event-Based Time-Surfaces for Pattern Recognition

Spatio-Temporal Backpropagation for Training High-Performance Spiking Neural Networks

CIFAR10-DVS: An Event-Stream Dataset for Object Classification

Ensemble Adversarial Training: Attacks and Defenses

Neural Style Transfer: A Review

Improved Training of Wasserstein GANs

Theory and Tools for the Conversion of Analog to Spiking Convolutional Neural Networks

Training Deep Spiking Neural Networks Using Backpropagation

SGDR: Stochastic Gradient Descent with Warm Restarts

Image Style Transfer Using Convolutional Neural Networks

Conversion of artificial recurrent neural networks to spiking neural networks for low-power neuromorphic hardware

Deep Residual Learning for Image Recognition

Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades

Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing

Domain-Adversarial Training of Neural Networks

U-Net: Convolutional Networks for Biomedical Image Segmentation

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Very Deep Convolutional Networks for Large-Scale Image Recognition

Wikipedia

Differentiable Spike: Rethinking Gradient-Descent for Training Spiking Neural Networks

Event camera — Wikipedia, the free encyclopedia

contributors: Spikingjelly

The Next Generation Neural Networks : Deep Learning and Spiking Neural Networks

Dropout: a simple way to prevent neural networks from overfitting

Neuromorphic Data

We conduct extensive experiments to verify the effectiveness of our proposed NDA on several benchmark DVS

Field of Study