N-ImageNet: Towards Robust, Fine-Grained Object Recognition with Event Cameras (2021-10-01T00:00:00.000000Z)

TL;DR

It is empirically show that pretraining on N-ImageNet improves the performance of event-based classifiers and helps them learn with few labeled data, and is the first to quantitatively investigate the consequences caused by various environmental conditions on event- based object recognition algorithms.

Abstract

We introduce N-ImageNet, a large-scale dataset targeted for robust, fine-grained object recognition with event cameras. The dataset is collected using programmable hardware in which an event camera consistently moves around a monitor displaying images from ImageNet. N-ImageNet serves as a challenging benchmark for event-based object recognition, due to its large number of classes and samples. We empirically show that pretraining on N-ImageNet improves the performance of event-based classifiers and helps them learn with few labeled data. In addition, we present several variants of N-ImageNet to test the robustness of event-based classifiers under diverse camera trajectories and severe lighting conditions, and propose a novel event representation to alleviate the performance degradation. To the best of our knowledge, we are the first to quantitatively investigate the consequences caused by various environmental conditions on event-based object recognition algorithms. N-ImageNet and its variants are expected to guide practical implementations for deploying event-based object recognition algorithms in the real world.

Authors

Jaehyeok Bae

1 papers

Gang-Ryeong Park

1 papers

Y. Kim

3 papers

TL;DR

Abstract

Authors

References64 items

Sparse Convolutions on Continuous Domains for Point Cloud and Event Stream Networks

Introduction and Analysis of an Event-Based Sign Language Dataset

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Temporal Binary Representation for Event-Based Action Recognition

On-Device Event Filtering with Binary Neural Networks for Pedestrian Detection Using Neuromorphic Vision Sensors

Learning to Detect Objects with a 1 Megapixel Event Camera

Unsupervised Feature Learning for Event Data: Direct vs Inverse Problem Formulation

Event Enhanced High-Quality Image Recovery

V2E: From video frames to realistic DVS event camera streams

Joint Filtering of Intensity Images and Neuromorphic Events for High-Resolution Noise-Robust Imaging

Denoising of Event-Based Sensors with Spatial-Temporal Correlation

Convolutional Spiking Neural Networks for Spatio-Temporal Feature Extraction

Learning to Exploit Multiple Vision Modalities by Using Grafted Networks

Event-based Asynchronous Sparse Convolutional Networks

EventSR: From Asynchronous Events to Image Reconstruction, Restoration, and Super-Resolution via End-to-End Adversarial Learning

Event Density Based Denoising Method for Dynamic Vision Sensor

Low-Power Dynamic Object Detection and Classification With Freely Moving Event Cameras

Residual

A Large Scale Event-based Detection Dataset for Automotive

A Differentiable Recurrent Surface for Asynchronous Event-Based Data

Video to Events: Recycling Video Datasets for Event Cameras

Graph-Based Object Classification for Neuromorphic Vision Sensing

EV-Gait: Event-Based Robust Gait Recognition Using Dynamic Vision Sensors

DET: A High-Resolution DVS Dataset for Lane Extraction

End-to-End Learning of Representations for Asynchronous Event-Based Data

Event-Based High Dynamic Range Image and Very High Frame Rate Video Generation Using Conditional Generative Adversarial Networks

Continuous-time Intensity Estimation Using Event Cameras

ACE: An Efficient Asynchronous Corner Tracker for Event Cameras

PRED18: Dataset and Further Experiments with DAVIS Event Camera in Predator-Prey Robot Chasing

Event-Based Vision Meets Deep Learning on Steering Prediction for Self-Driving Cars

HATS: Histograms of Averaged Time Surfaces for Robust Event-Based Object Classification

EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras

Spatial and Temporal Downsampling in Event-Based Visual Classification

A Low Power, Fully Event-Based Gesture Recognition System

HOTS: A Hierarchy of Event-Based Time-Surfaces for Pattern Recognition

Temperature and Parasitic Photocurrent Effects in Dynamic Vision Sensors

CIFAR10-DVS: An Event-Stream Dataset for Object Classification

4.1 A 640×480 dynamic vision sensor with a 9µm pixel and 300Meps address-event representation

Live demonstration: Convolutional neural network driven by dynamic vision sensor playing RoShamBo

Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences

Performance improvement of deep learning based gesture recognition using spatiotemporal demosaicing technique

Training Deep Spiking Neural Networks Using Backpropagation

DVS Benchmark Datasets for Object Tracking, Action Recognition, and Object Recognition

Evaluation of Event-Based Algorithms for Optical Flow with Ground-Truth from Inertial Measurement Sensor

Poker-DVS and MNIST-DVS. Their History, How They Were Made, and Other Details

Deep Residual Learning for Image Recognition

HFirst: A Temporal Approach to Object Recognition

Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades

Show, Attend and Tell: Neural Image Caption Generation with Visual Attention

ImageNet Large Scale Visual Recognition Challenge

Microsoft COCO: Common Objects in Context

A spiking neural network architecture for visual motion estimation

ImageNet classification with deep convolutional neural networks

Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories

Long Short-Term Memory

Learning to See in the Dark with Events

Learning Event-Driven Video Deblurring and Interpolation

and Gesture Recognition (FG 2020) (FG)

Getting Started with Arduino

O(n)-space spatiotemporal filter for reducing noise in neuromorphic vision sensors

Mapping from Frame-Driven to Frame-Free Event-Driven Vision Systems by Low-Rate Rate Coding and Coincidence Processing--Application to Feedforward ConvNets

Mnist handwritten digit database

FOR LOW POWER

and as an in

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names