A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction (2017-04-07T00:00:00.000000Z)

TL;DR

A dual-stage attention-based recurrent neural network (DA-RNN) to address the long-term temporal dependencies of the Nonlinear autoregressive exogenous model and can outperform state-of-the-art methods for time series prediction.

Abstract

The Nonlinear autoregressive exogenous (NARX) model, which predicts the current value of a time series based upon its previous values as well as the current and past values of multiple driving (exogenous) series, has been studied for decades. Despite the fact that various NARX models have been developed, few of them can capture the long-term temporal dependencies appropriately and select the relevant driving series to make predictions. In this paper, we propose a dual-stage attention-based recurrent neural network (DA-RNN) to address these two issues. In the first stage, we introduce an input attention mechanism to adaptively extract relevant driving series (a.k.a., input features) at each time step by referring to the previous encoder hidden state. In the second stage, we use a temporal attention mechanism to select relevant encoder hidden states across all time steps. With this dual-stage attention scheme, our model can not only make predictions effectively, but can also be easily interpreted. Thorough empirical studies based upon the SML 2010 dataset and the NASDAQ 100 Stock dataset demonstrate that the DA-RNN can outperform state-of-the-art methods for time series prediction.

Authors

Wei Cheng

2 papers

Haifeng Chen

2 papers

Guofei Jiang

2 papers

TL;DR

Abstract

Authors

References55 items

Applied Econometrics

Adaptive Feature Abstraction for Translating Video to Text

Adaptive Feature Abstraction for Translating Video to Language

Fast Structural Binary Coding

Hierarchical Attention Networks for Document Classification

TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems

Top Rank Supervised Binary Coding for Visual Search

Saliency detection via Cellular Automata

Show, Attend and Tell: Neural Image Caption Generation with Visual Attention

A Regularized Linear Dynamical System Framework for Multivariate Time Series Analysis

Adam: A Method for Stochastic Optimization

Deep visual-semantic alignments for generating image descriptions

Sequence to Sequence Learning with Neural Networks

On the Properties of Neural Machine Translation: Encoder–Decoder Approaches

Neural Machine Translation by Jointly Learning to Align and Translate

Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

Recurrent Continuous Translation Models

Time Series: Theory and Methods

Substructure Vibration NARX Neural Network Approach for Statistical Damage Inference

Dynamic Covariance Models for Multivariate Financial Time Series

Speech recognition with deep recurrent neural networks

Integrated pre-processing for Bayesian nonlinear system identification with Gaussian processes

Volume 3

Fine-Grained Photovoltaic Output Prediction Using a Bayesian Ensemble

Time Series: Theory and Methods (2nd ed,)

A dual-stage two-phase model of selective attention.

Et al

The use of NARX neural networks to predict chaotic time series

NARMAX time series model prediction: feedforward and recurrent fuzzy neural network approaches

ARMA Time-Series Modeling with Graphical Models

Proceedings of the 20th conference on Uncertainty in artificial intelligence

Long Short-Term Memory

Computational capabilities of recurrent NARX neural networks

Learning long-term dependencies in NARX recurrent neural networks

Experience with selecting exemplars from clean data

Neural networks

Learning long-term dependencies with gradient descent is difficult

Distributed representations, simple recurrent networks, and grammatical structure

Backpropagation Through Time: What It Does and How to Do It

Learning representations by back-propagating errors

Hypothesis Testing in Time Series Analysis.

ARIMA Models and the Box-Jenkins Methodology

In IJCAI

In NIPS

Brockwell and Richard A Davis

NARX-Based Nonlinear System Identification Using Orthogonal Least Squares Basis Hunting

Ensemble Learning for Time Series Prediction

IEEE Transactions on Neural Networks

Psychological Review

and Cybernetics

simple recurrent networks

Neural Computation

Nature

Journal of Engineering Mechanics

Hypothesis testing in time series analysis

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names