CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation (2021-07-07T00:00:00.000000Z)

TL;DR

Conditional Score-based Diffusion models for Imputation (CSDI), a novel time series imputation method that utilizes score-based diffusion models conditioned on observed data, which improves by 40-65% over existing probabilistic imputation methods on popular performance metrics.

Abstract

The imputation of missing values in time series has many applications in healthcare and finance. While autoregressive models are natural candidates for time series imputation, score-based diffusion models have recently outperformed existing counterparts including autoregressive models in many tasks such as image generation and audio synthesis, and would be promising for time series imputation. In this paper, we propose Conditional Score-based Diffusion models for Imputation (CSDI), a novel time series imputation method that utilizes score-based diffusion models conditioned on observed data. Unlike existing score-based approaches, the conditional diffusion model is explicitly trained for imputation and can exploit correlations between observed values. On healthcare and environmental data, CSDI improves by 40-65% over existing probabilistic imputation methods on popular performance metrics. In addition, deterministic imputation by CSDI reduces the error by 5-20% compared to the state-of-the-art deterministic imputation methods. Furthermore, CSDI can also be applied to time series interpolation and probabilistic forecasting, and is competitive with existing baselines. The code is available at https://github.com/ermongroup/CSDI.

Authors

Stefano Ermon

21 papers

Y. Tashiro

1 papers

Jiaming Song

2 papers

TL;DR

Abstract

Authors

References45 items

Generative Semi-supervised Learning for Multivariate Time Series Imputation

Symbolic Music Generation with Diffusion Models

Improved Denoising Diffusion Probabilistic Models

Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting

Temporal Latent Auto-Encoder: A Method for Probabilistic Multivariate Time Series Forecasting

GLIMA: Global and Local Time Series Imputation with Multi-directional Attention Learning

Score-Based Generative Modeling through Stochastic Differential Equations

RDIS: Random Drop Imputation With Self-Training for Incomplete Time Series Data

Denoising Diffusion Implicit Models

DiffWave: A Versatile Diffusion Model for Audio Synthesis

WaveGrad: Estimating Gradients for Waveform Generation

Solving Linear Inverse Problems Using the Prior Implicit in a Denoiser

Denoising Diffusion Probabilistic Models

Improved Techniques for Training Score-Based Generative Models

Multi-Time Attention Networks for Irregularly Sampled Time Series

Permutation Invariant Graph Generation via Score-Based Generative Modeling

Transformer Hawkes Process

Multi-variate Probabilistic Time Series Forecasting via Conditioned Normalizing Flows

PyTorch: An Imperative Style, High-Performance Deep Learning Library

High-Dimensional Multivariate Forecasting with Low-Rank Gaussian Copula Processes

Uncertainty-Aware Variational-Recurrent Imputation Network for Clinical Time Series

E²GAN: End-to-End Generative Adversarial Network for Multivariate Time Series Imputation

Generative Modeling by Estimating Gradients of the Data Distribution

GP-VAE: Deep Probabilistic Time Series Imputation

Efficient Attention: Attention with Linear Complexities

GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration

BRITS: Bidirectional Recurrent Imputation for Time Series

Estimating Missing Data in Temporal Data Streams Using Multi-Directional Recurrent Neural Networks

Attention is All you Need

Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks

ST-MVL: Filling Missing Values in Geo-Sensory Time Series Data

Recurrent Neural Networks for Multivariate Time Series with Missing Values

Deep Unsupervised Learning using Nonequilibrium Thermodynamics

Predicting in-hospital mortality of ICU patients: The PhysioNet/Computing in cardiology challenge 2012

MICE: Multivariate Imputation by Chained Equations in R

A Gradient Descent Algorithm on the Grassman Manifold for Matrix Completion

Nearest neighbor imputation of species-level, plot-scale forest structure attributes from LiDAR data

Multi-task Gaussian Process Prediction

Missing data: A comparison of neural network and expectation maximization techniques

Scoring Rules for Continuous Probability Distributions

GluonTS: Probabilistic and Neural Time Series Modeling in Python

Linear attention transformer

Latent Ordinary Differential Equations for Irregularly-Sampled Time Series

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Multivariate Time Series Imputation with Generative Adversarial Networks

Field of Study

Venue Information

Name

Type

URL

Alternate Names