The neural moving average model for scalable variational inference of state space models (2019-10-02T00:00:00.000000Z)

TL;DR

This work proposes an extension to state space models of time series data based on a novel generative model for latent temporal states: the neural moving average model, which permits a subsequence to be sampled without drawing from the entire distribution, enabling training iterations to use mini-batches of the time series at low computational cost.

Abstract

Variational inference has had great success in scaling approximate Bayesian inference to big data by exploiting mini-batch training. To date, however, this strategy has been most applicable to models of independent data. We propose an extension to state space models of time series data based on a novel generative model for latent temporal states: the neural moving average model. This permits a subsequence to be sampled without drawing from the entire distribution, enabling training iterations to use mini-batches of the time series at low computational cost. We illustrate our method on autoregressive, Lotka-Volterra, FitzHugh-Nagumo and stochastic volatility models, achieving accurate parameter estimation in a short time.

Authors

Tom Ryder

1 papers

D. Prangle

1 papers

A. Golightly

1 papers

TL;DR

Abstract

Authors

References52 items

BayesFlow: Learning Complex Stochastic Models With Invertible Neural Networks

Normalizing Flows for Probabilistic Modeling and Inference

Black-Box Inference for Non-Linear Latent Force Models

Neural Spline Flows

Applied Stochastic Differential Equations

MaCow: Masked Convolutional Generative Flow

Stochastic Gradient MCMC for State Space Models

Scalable Bayesian Learning for State Space Models using Variational Inference with SMC Samplers

Black-Box Variational Inference for Stochastic Differential Equations

Parallel WaveNet: Fast High-Fidelity Speech Synthesis

Masked Autoregressive Flow for Density Estimation

Fast and Scalable Gaussian Process Modeling with Applications to Astronomical Time Series

Structured Inference Networks for Nonlinear State Space Models

Conditional Image Generation with PixelCNN Decoders

Improved Variational Inference with Inverse Autoregressive Flow

Density estimation using Real NVP

Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data

Variational Inference: A Review for Statisticians

BLACK BOX VARIATIONAL INFERENCE FOR STATE SPACE MODELS

Improved bridge constructs for stochastic differential equations

Variational Inference with Normalizing Flows

MADE: Masked Autoencoder for Distribution Estimation

Adam: A Method for Stochastic Optimization

A comparison of inferential methods for highly nonlinear state space models in ecology and epidemiology

Stochastic variational inference for hidden Markov models

Doubly Stochastic Variational Bayes for non-Conjugate Inference

Bayesian estimation of discretely observed multi-dimensional diffusion processes using guided proposals

Stochastic Backpropagation and Approximate Inference in Deep Generative Models

Auto-Encoding Variational Bayes

On the efficiency of pseudo-marginal random walk Metropolis algorithms

Bayesian Filtering and Smoothing

Inference for Diffusion Processes: With Applications in Life Sciences

On the difficulty of training recurrent neural networks

A Kernel Two-Sample Test

Bayesian parameter inference for stochastic biochemical network models using particle Markov chain Monte Carlo

A new look at state-space models for neural data

Smoothing algorithms for state–space models

Bayesian sequential inference for nonlinear multivariate diffusions

Inference in Hidden Markov Models

Numerical Techniques for Maximum Likelihood Estimation of Continuous-Time Diffusion Processes

MCMC Analysis of Diffusion Models With Application to Finance

An Introduction to Variational Methods for Graphical Models

Estimating continuous-time stochastic volatility models of the short-term interest rate

Reversible jump Markov chain Monte Carlo computation and Bayesian model determination

Hidden Markov Models: Estimation and Control

A tutorial on hidden Markov models and selected applications in speech recognition

Stochastic differential equations : an introduction with applications

Bayesian Forecasting And Dynamic Models

State-space models : applications in economics and finance

Markov chain Monte Carlo approach to parameter estimation in the FitzHughNagumo model

Bayesian inference for nonlinear multivariate diffusion models observed with error

Convergence of Sequential Monte Carlo Methods

Field of Study

Venue Information

Name

Type

URL

Alternate Names