Dynamic Measurement Scheduling for Event Forecasting using Deep RL (2019-01-24T00:00:00.000000Z)

TL;DR

Deep reinforcement learning is used to answer the question of what and when to be measured to forecast detrimental events, by scheduling strategically-timed measurements that jointly minimizes the measurement cost and maximizes predictive gain.

Abstract

Imagine a patient in critical condition. What and when should be measured to forecast detrimental events, especially under the budget constraints? We answer this question by deep reinforcement learning (RL) that jointly minimizes the measurement cost and maximizes predictive gain, by scheduling strategically-timed measurements. We learn our policy to be dynamically dependent on the patient's health history. To scale our framework to exponentially large action space, we distribute our reward in a sequential setting that makes the learning easier. In our simulation, our policy outperforms heuristic-based scheduling with higher predictive gain and lower cost. In a real-world ICU mortality prediction task (MIMIC3), our policies reduce the total number of measurements by $31\%$ or improve predictive gain by a factor of $3$ as compared to physicians, under the off-policy policy evaluation.

Authors

A. Goldenberg

5 papers

C. Chang

3 papers

Mingjie Mai

1 papers

TL;DR

Abstract

Authors

References48 items

Impact of Serum Phosphate in Mechanically Ventilated Patients With Severe Sepsis and Septic Shock

Assessment of Intensive Care Unit Laboratory Values That Differ From Reference Ranges and Association With Patient Mortality and Length of Stay

The Artificial Intelligence Clinician learns optimal treatment strategies for sepsis in intensive care

An Optimal Policy for Patient Laboratory Tests in Intensive Care Units

Explaining Image Classifiers by Counterfactual Generation

Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendation

The Association between Arterial Oxygen Tension, Hemoglobin Concentration, and Mortality in Mechanically Ventilated Critically Ill Patients

xGEMs: Generating Examplars to Explain Black-Box Models

Representation Balancing MDPs for Off-Policy Policy Evaluation

Confounding-Robust Policy Improvement

Deep Sensing: Active Sensing using Multi-directional Recurrent Neural Networks

Learning to Treat Sepsis with Multi-Output Gaussian Process Deep Recurrent Q-Networks

Overdiagnosis: what it is and what it isn’t

Representation and Reinforcement Learning for Personalized Glycemic Control in Septic Patients

Medical Diagnosis From Laboratory Tests by Combining Generative and Discriminative Learning

Why Pay More When You Can Pay Less: A Joint Learning Framework for Active Feature Acquisition and Classification

An Improved Multi-Output Gaussian Process RNN with Real-Time Validation for Early Sepsis Detection

Scalable Joint Models for Reliable Uncertainty-Aware Event Prediction

Reducing Unnecessary Laboratory Testing in the Medical ICU.

Continuous State-Space Models for Optimal Sepsis Treatment: a Deep Reinforcement Learning Approach

A Reinforcement Learning Approach to Weaning of Mechanical Ventilation in Intensive Care Units

What-If Reasoning with Counterfactual Gaussian Processes

Multitask learning and benchmarking with clinical time series data

Sequential Cost-Sensitive Feature Acquisition

A scalable end-to-end Gaussian process adapter for irregularly sampled time series classification

Modeling Missing Data in Clinical Time Series with RNNs

MIMIC-III, a freely accessible critical care database

Active Information Acquisition

Dueling Network Architectures for Deep Reinforcement Learning

Doubly Robust Off-policy Value Evaluation for Reinforcement Learning

Using information theory to identify redundancy in common laboratory tests in the intensive care unit

A Multivariate Timeseries Modeling Approach to Severity of Illness Assessment and Forecasting in ICU with Sparse, Heterogeneous Clinical Data

Arterial hyperoxia and mortality in critically ill patients: a systematic review and meta-analysis

Developing Predictive Models Using Electronic Medical Records: Challenges and Pitfalls

Embedding Time-Limited Laboratory Orders Within Computerized Provider Order Entry Reduces Laboratory Utilization*

Overdiagnosis of disease: a modern epidemic.

Preventing overdiagnosis: how to stop harming the healthy

Diagnostic blood loss from phlebotomy and hospital-acquired anemia during acute myocardial infarction.

Treatment of hypophosphatemia in the intensive care unit: a review

Managing the cost of diagnosis.

Laboratory testing in the intensive care unit.

Long Short-Term Memory

Reducing Unnecessary Postoperative Complete Blood Count Testing in the Pediatric Intensive Care Unit.

DPSCREEN: Dynamic Personalized Screening

Treatmentresponse models for counterfactual reasoning with continuous-time, continuous-valued interventions

Preventing over-diagnosis – how to stop harming the healthy

Implementation of an on-demand strategy for routine blood testing in icu patients

Nosocomial anemia.

Field of Study

Venue Information

Name

Type

URL

Alternate Names