Kitsune: An Ensemble of Autoencoders for Online Network Intrusion Detection (2018-02-25T00:00:00.000000Z)

TL;DR

Kitsune is presented: a plug and play NIDS which can learn to detect attacks on the local network, without supervision, and in an efficient online manner, and demonstrates that Kitsune can be a practical and economic NIDS.

Abstract

Neural networks have become an increasingly popular solution for network intrusion detection systems (NIDS). Their capability of learning complex patterns and behaviors make them a suitable solution for differentiating between normal traffic and network attacks. However, a drawback of neural networks is the amount of resources needed to train them. Many network gateways and routers devices, which could potentially host an NIDS, simply do not have the memory or processing power to train and sometimes even execute such models. More importantly, the existing neural network solutions are trained in a supervised manner. Meaning that an expert must label the network traffic and update the model manually from time to time. In this paper, we present Kitsune: a plug and play NIDS which can learn to detect attacks on the local network, without supervision, and in an efficient online manner. Kitsune's core algorithm (KitNET) uses an ensemble of neural networks called autoencoders to collectively differentiate between normal and abnormal traffic patterns. KitNET is supported by a feature extraction framework which efficiently tracks the patterns of every network channel. Our evaluations show that Kitsune can detect various attacks with a performance comparable to offline anomaly detectors, even on a Raspberry PI. This demonstrates that Kitsune can be a practical and economic NIDS.

Authors

A. Shabtai

8 papers

Y. Elovici

10 papers

Yisroel Mirsky

2 papers

TL;DR

Abstract

Authors

References52 items

Adversarial machine learning

Gaussian Mixture Models

Autoencoder-based feature learning for cyber security applications

A Survey of Data Mining and Machine Learning Methods for Cyber Security Intrusion Detection

Enhanced situation space mining for data streams

A Time Aware Method for Predicting Dull Nodes and Links in Evolving Networks for Data Cleaning

An Uncertain Future: Forecasting from Static Images Using Variational Autoencoders

A Deep Learning Approach for Network Intrusion Detection System

An Empirical Analysis of Cyber Security Incidents at a Large Organization

Training Very Deep Networks

Anomaly Detection Using Autoencoders with Nonlinear Dimensionality Reduction

An ensemble of deep neural networks for object tracking

Data stream clustering: A survey

Scalable Hypergrid k-NN-Based Online Anomaly Detection in Wireless Sensor Networks

Novel intrusion detection system integrating layered framework with neural network

A Review of Machine Learning based Anomaly Detection Techniques

Survey on data mining techniques to enhance intrusion detection

Methods of Hierarchical Clustering

Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion

A Small Sphere and Large Margin Approach for Novelty Detection Using Training Data with Outliers

Anomaly detection: A survey

Anomaly-based network intrusion detection: Techniques, systems and challenges

Isolation Forest

A hybrid machine learning approach to network anomaly detection

Incremental learning of gestures by imitation in a humanoid robot

A machine learning framework for network anomaly detection using SVM and GA

Intrusion detection using hierarchical neural networks

Intrusion detection using an ensemble of intelligent paradigms

Anomalous Payload-Based Network Intrusion Detection

Extreme learning machine: a new learning scheme of feedforward neural networks

Data Mining Approaches for Intrusion Detection

Neural network design

Suricata — open source ids / ips / nsm engine

Index of /open/

Data Mining for Network Intrusion Detection : A comparison of data mining algorithms and an analysis of relevant features for detecting cyber-attacks

Instant Trafﬁc Analysis with Tshark How-to

Evaluation of anomaly-based IDS for mobile devices using machine learning classifiers

Stochastic Gradient Descent Tricks

An efficient intrusion detection system based on support vector machines and gradually feature removal method

An Enhanced Resilient Backpropagation Artificial Neural Network for Intrusion Detection System

Statistical Normalization and Back Propagationfor Classification

Optimization for machine learning

Design and Implementation of a Network Emulator using Virtual Network Stack

Guarantee that each ~v i has no more than m features, where m is a user deﬁned parameter of the system

Contain subspaces of X which capture the normal behavior well enough to detect anomalous events occurring in the respective subspaces

The Packet Parser receives the raw binary, parses the packet, and sends the meta information of the packet to the FE. For example

Train the ANN to classify the difference between normal and attack trafﬁc, using a strong CPU or GPU

Suricata idps and its interaction with linux kernel

Transfer a copy of the trained model to the network/organization’s NIDS

Map each of the n features in ~x exactly once to the features in v . This is to ensure that the ensemble is not too wide

The FE receives this information, and uses it to retrieve over 100 statistics which are used to implicitly describe the current state of the channel from which the packet came

Machine learning based botnet detection us-ing real-time extracted trafﬁc features, March 25 2014

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names