Deep Learning for Anomaly Detection: A Survey

Published in

arXiv.org(2019)

External Links:

Generate Graph

TL;DR

A structured and comprehensive overview of research methods in deep learning-based anomaly detection, grouped state-of-the-art research techniques into different categories based on the underlying assumptions and approach adopted.

Abstract

Anomaly detection is an important problem that has been well-studied within diverse research areas and application domains. The aim of this survey is two-fold, firstly we present a structured and comprehensive overview of research methods in deep learning-based anomaly detection. Furthermore, we review the adoption of these methods for anomaly across various application domains and assess their effectiveness. We have grouped state-of-the-art research techniques into different categories based on the underlying assumptions and approach adopted. Within each category we outline the basic anomaly detection technique, along with its variants and present key assumptions, to differentiate between normal and anomalous behavior. For each category, we present we also present the advantages and limitations and discuss the computational complexity of the techniques in real application domains. Finally, we outline open issues in research and challenges faced while adopting these techniques.

Authors

Raghavendra Chalapathy

2 papers

Sanjay Chawla

1 papers

References477 items

DeepFall: Non-Invasive Fall Detection with Deep Spatio-Temporal Convolutional Autoencoders

Packet-data anomaly detection in PMU-based state estimator using convolutional neural network

Artificial Intelligence Crime: An Interdisciplinary Analysis of Foreseeable Threats and Solutions

Aircraft engine fault detection based on grouped convolutional denoising autoencoders

MAD-GAN: Multivariate Anomaly Detection for Time Series Data with Generative Adversarial Networks

Deep Learning for Anomaly Detection: A Survey

Published in

arXiv.org(2019)

External Links:

Generate Graph

TL;DR

Abstract

Authors

Raghavendra Chalapathy

2 papers

Sanjay Chawla

1 papers

References477 items

DeepFall: Non-Invasive Fall Detection with Deep Spatio-Temporal Convolutional Autoencoders

Packet-data anomaly detection in PMU-based state estimator using convolutional neural network

Artificial Intelligence Crime: An Interdisciplinary Analysis of Foreseeable Threats and Solutions

Aircraft engine fault detection based on grouped convolutional denoising autoencoders

MAD-GAN: Multivariate Anomaly Detection for Time Series Data with Generative Adversarial Networks

AlphaMEX: A smarter global pooling method for convolutional neural networks

Attentioned Convolutional LSTM InpaintingNetwork for Anomaly Detection in Videos

A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Data

Estimation of Dimensions Contributing to Detected Anomalies with Variational Autoencoders

Multidimensional Time Series Anomaly Detection: A GRU-based Gaussian Mixture Variational Autoencoder Approach

MuVAN: A Multi-view Attention Network for Multivariate Temporal Data

Artificial intelligence in retina

Exploring Adversarial Examples in Malware Detection

MDGAN: Boosting Anomaly Detection Using Multi-Discriminator Generative Adversarial Networks

Generative Ensembles for Robust Anomaly Detection

WAIC, but Why? Generative Ensembles for Robust Anomaly Detection

Flow-based Network Traffic Generation using Generative Adversarial Networks

Exploring the interpretability of LSTM neural networks over multi-variable data

Deep Learning Approach Combining Sparse Autoencoder With SVM for Network Intrusion Detection

Long-term temporal averaging for stochastic optimization of deep neural networks

Toward Sustainable Virtualized Healthcare: Extracting Medical Entities from Chinese Online Health Consultations Using Deep Neural Networks

Anomaly Detection with Generative Adversarial Networks for Multivariate Time Series

Deep learning for time series classification: a review

Host Based Intrusion Detection System with Combined CNN/RNN Model

Image Anomaly Detection with Generative Adversarial Networks

BINet: Multivariate Business Process Anomaly Detection Using Deep Learning

VOS: a Method for Variational Oversampling of Imbalanced Data

Semi-Supervised Outlier Detection Using a Generative and Adversary Framework

Anomaly Detection in Moving Crowds through Spatiotemporal Autoencoding and Additional Attention

Zero-day malware detection using transferred generative adversarial networks based on deep autoencoders

Evaluation of deep learning approaches based on convolutional neural networks for corrosion detection

Scalable Deep Learning through Fuzzy-Based Clustering in Autonomous Systems

MULDEF: Multi-model-based Defense Against Adversarial Examples for Neural Networks

Using convolutional neural networks for classification of malware represented as images

An investigation of a deep learning based malware detection system

DOPING: Generative Data Augmentation for Unsupervised Anomaly Detection with GAN

Enhanced Network Anomaly Detection Based on Deep Neural Networks

A Model Based on Convolutional Neural Network for Online Transaction Fraud Detection

A deep Recurrent Neural Network based approach for Internet of Things malware threat hunting

A Combined Deep Learning GRU-Autoencoder for the Early Detection of Respiratory Disease in Pigs Using Multiple Environmental Sensors

Credit Card Fraud Detection Based on Whale Algorithm Optimized BP Neural Network

A Survey of Anomaly Detection in Industrial Wireless Sensor Networks with Critical Water System Infrastructure as a Case Study

Detecting Anomaly in Big Data System Logs Using Convolutional Neural Network

Semi-Supervised Generative Adversarial Network for Gene Expression Inference

Are generative deep models for novelty detection truly better?

When deep learning meets security

Automatic Rumor Detection on Microblogs: A Survey

Spatio-Temporal Network Traffic Estimation and Anomaly Detection Based on Convolutional Neural Network in Vehicular Ad-Hoc Networks

Recurrent Neural Networks for Enhancement of Signature-based Network Intrusion Detection Systems

AND: Autoregressive Novelty Detectors

Deep One-Class Classification

Anomaly Detection for Skin Disease Images Using Variational Autoencoder

Deep learning at the shallow end: Malware classification for non-domain experts

A Deep Learning-Based Method for Sleep Stage Classification Using Physiological Signal

Anomaly Machine Component Detection by Deep Generative Model with Unregularized Score

A review on the application of deep learning in system health management

Time Series Prediction and Anomaly Detection of Light Curve Using LSTM Neural Network

Malware Detection in Cloud Infrastructures Using Convolutional Neural Networks

Toward Explainable Deep Neural Network Based Anomaly Detection

An Empirical Study on Network Anomaly Detection Using Convolutional Neural Networks

LiReD: A Light-Weight Real-Time Fault Detection System for Edge Computing Using LSTM Recurrent Neural Networks

Neural Network Cognitive Engine for Autonomous and Distributed Underlay Dynamic Spectrum Access

A Review of Artificial Intelligence Algorithms Used for Smart Machine Tools

Desh: deep learning for system health prediction of lead times to failure in HPC

Insider Threat Detection with Deep Neural Network

SU-IDS: A Semi-supervised and Unsupervised Framework for Network Intrusion Detection

Probabilistic Deep Learning using Random Sum-Product Networks

DeepAD: A Generic Framework Based on Deep Learning for Time Series Anomaly Detection

Generative adversarial network based telecom fraud detection at the receiving bank

Outlier Detection for Multidimensional Time Series Using Deep Neural Networks

Unsupervised Anomaly Detection for Traffic Surveillance Based on Background Modeling

Sequence classification for credit-card fraud detection

Anomaly Detection for Short Texts: Identifying Whether Your Chatbot Should Switch from Goal-Oriented Conversation to Chit-Chatting

Semi-supervised Encoding for Outlier Detection in Clinical Observation Data

An enhancing framework for botnet detection using generative adversarial networks

A hybrid deep learning model for consumer credit scoring

Detecting Deceptive Reviews Using Generative Adversarial Networks

Doctor AI: Interpretable deep learning for modeling electronic health records

Distributed Anomaly Detection Using Autoencoder Neural Networks in WSN for IoT

Critical Analysis of Machine Learning Based Approaches for Fraud Detection in Financial Transactions

GANomaly: Semi-Supervised Anomaly Detection via Adversarial Training

Detecting and Locating Gastrointestinal Anomalies Using Deep Learning and Iterative Cluster Unification

Sequence to Sequence Pattern Learning Algorithm for Real-Time Anomaly Detection in Network Traffic

End-to-End Deep Neural Networks and Transfer Learning for Automatic Analysis of Nation-State Malware

Towards Experienced Anomaly Detector Through Reinforcement Learning

Effective android malware detection with a hybrid model based on deep autoencoder and convolutional neural network

Residual Error Based Anomaly Detection Using Auto-Encoder in SMD Machine Sound

Stan: Spatio- Temporal Adversarial Networks for Abnormal Event Detection

An Unsupervised Anomalous Event Detection Framework with Class Aware Source Separation

Adversarial Attacks Against Medical Deep Learning Systems

Group Anomaly Detection using Deep Generative Models

Unsupervised Detection of Lesions in Brain MRI using constrained adversarial auto-encoders

LSTM-Based Analysis of Industrial IoT Equipment

Deep belief network based detection and categorization of malicious URLs

Improved anomaly detection in surveillance videos based on a deep learning method

End-to-end neural network architecture for fraud scoring in card payments

Malware classification using deep learning methods

Convolutional neural networks for crowd behaviour analysis: a survey

Detecting Urban Anomalies Using Multiple Spatio-Temporal Data Sources

100

Residential roof condition assessment system using deep learning

101

Recurrent Neural Network Attention Mechanisms for Interpretable System Log Anomaly Detection

102

Adversarial Malware Binaries: Evading Deep Learning for Malware Detection in Executables

103

A Compact Convolutional Neural Network for Textured Surface Anomaly Detection

104

One-Class Adversarial Nets for Fraud Detection

105

An Empirical Evaluation of Deep Learning for Network Anomaly Detection

106

Analyzing business process anomalies using autoencoders

107

MalDozer: Automatic framework for android malware detection using deep learning

108

Semi-Supervised Deep Learning Using Pseudo Labels for Hyperspectral Image Classification

109

A primitive study on unsupervised anomaly detection with an autoencoder in emergency head CT volumes

110

An Unsupervised Deep Hyperspectral Anomaly Detector

111

Adversarially Learned One-Class Classifier for Novelty Detection

112

Kitsune: An Ensemble of Autoencoders for Online Network Intrusion Detection

113

Anomaly Detection using One-Class Neural Networks

114

Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection

115

Detecting Anomalous Faces with 'No Peeking' Autoencoders

116

Novelty Detection with GAN

117

Jiffy: A Convolutional Approach to Learning Time Series Similarity

118

Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding

119

Efficient GAN-Based Anomaly Detection

120

Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications

121

Deep Learning and Its Applications in Biomedicine

122

Clustering and Unsupervised Anomaly Detection with l2 Normalized Deep Auto-Encoder Representations

123

RETRACTED ARTICLE: Detecting anomalous emotion through big data from social networks based on a deep learning method

124

Detection of Video Anomalies Using Convolutional Autoencoders and One-Class Support Vector Machines

125

Phonocardiographic Sensing Using Deep Learning for Abnormal Heartbeat Detection

126

Learning Deep Features for One-Class Classification

127

Image denoising and restoration with CNN-LSTM Encoder Decoder with Direct Attention

128

Role-based Log Analysis Applying Deep Learning for Insider Threat Detection

129

Detecting Cyber Attacks in Industrial Control Systems Using Convolutional Neural Networks

130

A two-stage flow-based intrusion detection model for next-generation networks

131

Real-World Anomaly Detection in Surveillance Videos

132

Online anomaly detection with concept drift adaptation using recurrent neural networks

133

An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos

134

HeNet: A Deep Learning Approach on Intel® Processor Trace for Effective Exploit Detection

135

Anomaly Detection in Nanofibrous Materials by CNN-Based Self-Similarity

136

Improving Malware Detection Accuracy by Extracting Icon Information

137

Deep Learning for IoT Big Data and Streaming Analytics: A Survey

138

Using generative adversarial networks for improving classification effectiveness in credit card fraud detection

139

Recurrent Neural Network Language Models for Open Vocabulary Event-Level Cyber Anomaly Detection

140

Medicare Fraud Detection Using Machine Learning Methods

141

Using deep networks for fraud detection in the credit card transactions

142

Anomaly detection in smart grid based on encoder-decoder framework with recurrent neural network

143

Detecting event-related changes in organizational networks using optimized neural network models

144

Improving the Accuracy of Pre-trained Word Embeddings for Sentiment Analysis

145

Relief-Based Feature Selection: Introduction and Review

146

A Hybrid Semi-Supervised Anomaly Detection Model for High-Dimensional Data

147

Deep Clustering with Convolutional Autoencoders

148

Deep Sequential Models for Task Satisfaction Prediction

149

Transaction Fraud Detection Using GRU-centered Sandwich-structured Model

150

A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-Based Variational Autoencoder

151

Detecting anomalies in time series data via a deep learning algorithm combining wavelets, neural networks and Hilbert transform

152

Learning temporal representation of transaction amount for fraudulent transaction recognition using CNN, Stacked LSTM, and CNN-LSTM

153

Unsupervised real-time anomaly detection for streaming data

154

DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning

155

Convolutional Neural Networks for Unsupervised Anomaly Detection in Text Data

156

Unsupervised Anomaly Detection With LSTM Neural Networks

157

Deep Convolutional Neural Networks for Anomaly Event Classification on Distributed Systems

158

Experience Report: Log Mining Using Natural Language Processing and Application to Anomaly Detection

159

Generalized zero-shot learning for action recognition with web-scale video data

160

Solving the "false positives" problem in fraud prediction

161

A Deep Learning Approach for Intrusion Detection Using Recurrent Neural Networks

162

Deep Learning for Unsupervised Insider Threat Detection in Structured Cybersecurity Data Streams

163

Malware Detection by Eating a Whole EXE

164

A Revisit of Sparse Coding Based Anomaly Detection in Stacked RNN Framework

165

The prototype of the HL-LHC magnets monitoring system based on Recurrent Neural Networks and adaptive quantization

166

A survey of deep learning-based network anomaly detection

167

A Grassmannian Approach to Zero-Shot Learning for Network Intrusion Detection

168

Comprehensive survey of deep learning in remote sensing: theories, tools, and challenges for the community

169

Transfer Learning for Time Series Anomaly Detection

170

Session-Based Fraud Detection in Online E-Commerce Transactions Using Recurrent Neural Networks

171

Detection of Anomalies in Large Scale Accounting Data using Deep Autoencoder Networks

172

Anomaly Detection for a Water Treatment System Using Unsupervised Machine Learning

173

A New Deep Learning Approach for Anomaly Base IDS using Memetic Classifier

174

RNN-based Early Cyber-Attack Detection for the Tennessee Eastman Process

175

Deep and Confident Prediction for Time Series at Uber

176

Anomaly Detection using a Convolutional Winner-Take-All Autoencoder

177

A Generative Model for Zero Shot Learning Using Conditional Variational Autoencoders

178

A Robust Deep-Learning-Based Detector for Real-Time Tomato Plant Diseases and Pests Recognition

179

DOC: Deep Open Classification of Text Documents

180

Comparative study of word embedding methods in topic segmentation

181

Evaluating Convolutional Neural Network for Effective Mobile Malware Detection

182

Applying convolutional neural network for network intrusion detection

183

Abnormal event detection in videos using generative adversarial nets

184

Neural Networks for Financial Market Risk Classification

185

Perspective analysis of telecommunication fraud detection using data stream analytics and neural network classification based data mining

186

Deep Belief Networks used on High Resolution Multichannel Electroencephalography Data for Seizure Detection

187

Conditional Variational Autoencoder for Prediction and Feature Recovery Applied to Intrusion Detection in IoT

188

Transfer learning for aluminium extrusion electricity consumption anomaly detection via deep neural networks

189

Cleaning Method for Status Monitoring Data of Power Equipment Based on Stacked Denoising Autoencoders

190

Fake Twitter followers detection by denoising autoencoder

191

Malware Detection with Convolutional Neural Network Using Hardware Events

192

Deep convolutional player modeling on log and level data

193

Time Series Anomaly Detection; Detection of anomalous drops with limited features and sparse examples in noisy highly periodic data

194

Anomaly Detection with Robust Deep Autoencoders

195

A transfer learning framework for traffic video using neuro-fuzzy approach

196

Convolutional and recurrent neural networks for real-time data classification

197

Using deep autoencoders to learn robust domain-invariant representations for still-to-video face recognition

198

Improved Deep Embedded Clustering with Local Structure Preservation

199

Deep Learning: A Practitioner's Approach

200

Abnormal event detection based on deep autoencoder fusing optical flow

201