Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration (2021-02-08T00:00:00.000000Z)

TL;DR

This work simulates patients at admission time, when decision support can be especially valuable, and proposes *clinical outcome pre-training* to integrate knowledge about patient outcomes from multiple public sources and presents a simple method to incorporate ICD code hierarchy into the models.

Abstract

Outcome prediction from clinical text can prevent doctors from overlooking possible risks and help hospitals to plan capacities. We simulate patients at admission time, when decision support can be especially valuable, and contribute a novel *admission to discharge* task with four common outcome prediction targets: Diagnoses at discharge, procedures performed, in-hospital mortality and length-of-stay prediction. The ideal system should infer outcomes based on symptoms, pre-conditions and risk factors of a patient. We evaluate the effectiveness of language models to handle this scenario and propose *clinical outcome pre-training* to integrate knowledge about patient outcomes from multiple public sources. We further present a simple method to incorporate ICD code hierarchy into the models. We show that our approach improves performance on the outcome tasks against several baselines. A detailed analysis reveals further strengths of the model, including transferability, but also weaknesses such as handling of vital values and inconsistencies in the underlying data.

Authors

Betty van Aken

2 papers

Jens-Michalis Papaioannou

1 papers

M. Mayrdorfer

1 papers

TL;DR

Abstract

Authors

References48 items

Predicting Multiple ICD-10 Codes from Brazilian-Portuguese Clinical Notes

Big Bird: Transformers for Longer Sequences

Self-Attention Enhanced Patient Journey Understanding in Healthcare System

Experimental Evaluation and Development of a Silver-Standard for the MIMIC-III Clinical Coding Dataset

BERT-XML: Large Scale Automated ICD Coding Using BERT Pretraining

Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks

Longformer: The Long-Document Transformer

Pre-trained models for natural language processing: A survey

Is Language Modeling Enough? Evaluating Effective Embedding Combinations

Towards unstructured mortality prediction with free-text clinical notes

Ontological attention ensembles for capturing semantic concepts in ICD code prediction from clinical text

Leveraging Medical Literature for Section Prediction in Electronic Health Records

Do NLP Models Know Numbers? Probing Numeracy in Embeddings

Using Clinical Notes with Time Series Data for ICU Management

How Does BERT Answer Questions?: A Layer-Wise Analysis of Transformer Representations

MNN: Multimodal Attentional Neural Networks for Diagnosis Prediction

Toward a clinical text encoder: pretraining for clinical natural language processing with applications to substance misuse

What do you learn from context? Probing for sentence structure in contextualized word representations

ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission

Publicly Available Clinical BERT Embeddings

An Analysis of Attention over Clinical Notes for Predictive Tasks

SECTOR: A Neural Model for Coherent Topic Segmentation and Classification

BioBERT: a pre-trained biomedical language representation model for biomedical text mining

A question-entailment approach to question answering

MiME: Multilevel Medical Embedding of Electronic Health Records for Predictive Healthcare

Deep EHR: Chronic Disease Prediction Using Medical Notes

Patient representation learning and interpretable evaluation using clinical notes

Learning Tasks for Multitask Learning: Heterogenous Patient Populations in the ICU

What’s in a Note? Unpacking Predictive Value in Clinical Note Representations

A Neural Architecture for Automated ICD Coding

Generating Multi-label Discrete Patient Records using Generative Adversarial Networks

Factors affecting length of stay in teaching hospitals of a middle-income country

MIMIC-III, a freely accessible critical care database

Annotating longitudinal clinical narratives for de-identification: The 2014 i2b2/UTHealth corpus

Automated systems for the de-identification of longitudinal clinical narratives: Overview of 2014 i2b2/UTHealth shared task Track 1

Annotating temporal information in clinical narratives

Evaluating temporal relations in clinical text: 2012 i2b2 Challenge

Evaluating the state of the art in coreference resolution for electronic medical records

2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text

Extracting medication information from clinical text

Community annotation experiment for ground truth generation for the i2b2 medication challenge

Proceedings of the ACM Conference on Health, Inference, and Learning

Deep Patient Representation of Clinical Notes via Multi-Task Learning for Mortality Prediction.

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Viewpoint Paper: Identifying Patient Smoking Status from Medical Discharge Records

Viewpoint Paper: Evaluating the State-of-the-Art in Automatic De-identification

Neural Architecture

Conference on Knowledge Discovery & Data Mining, KDD 2018 , pages 802–810, London, UK

Field of Study

Venue Information

Name

Type

URL

Alternate Names