Automated Medical Coding on MIMIC-III and MIMIC-IV: A Critical Review and Replicability Study (2023-04-21T00:00:00.000000Z)

TL;DR

This paper reproduce, compare, and analyze state-of-the-art automated medical coding machine learning models, showing that several models underperform due to weak configurations, poorly sampled train-test splits, and insufficient evaluation.

Abstract

Medical coding is the task of assigning medical codes to clinical free-text documentation. Healthcare professionals manually assign such codes to track patient diagnoses and treatments. Automated medical coding can considerably alleviate this administrative burden. In this paper, we reproduce, compare, and analyze state-of-the-art automated medical coding machine learning models. We show that several models underperform due to weak configurations, poorly sampled train-test splits, and insufficient evaluation. In previous work, the macro F1 score has been calculated sub-optimally, and our correction doubles it. We contribute a revised model comparison using stratified sampling and identical experimental setups, including hyperparameters and decision boundary tuning. We analyze prediction errors to validate and falsify assumptions of previous works. The analysis confirms that all models struggle with rare codes, while long documents only have a negligible impact. Finally, we present the first comprehensive results on the newly released MIMIC-IV dataset using the reproduced models. We release our code, model parameters, and new MIMIC-III and MIMIC-IV training and evaluation pipelines to accommodate fair future comparisons.

Authors

Lars Maaløe

3 papers

Maria Maistro

2 papers

Joakim Edin

1 papers

TL;DR

Abstract

Authors

References52 items

A Review on Deep Neural Networks for ICD Coding

Automating the overburdened clinical coding system: challenges and next steps

MIMIC-IV, a freely accessible electronic health record dataset

Knowledge Injected Prompt Based Fine-tuning for Multi-label Few-shot ICD Coding

PLM-ICD: Automatic ICD Coding with Pretrained Language Models

Self-Supervised Speech Representation Learning: A Review

ICDBigBird: A Contextual Embedding Model for ICD Code Classification

Automated clinical coding: what, why, and where we are?

Code Synonyms Do Matter: Multiple Synonyms Matching Network for Automatic ICD Coding

A Unified Review of Deep Learning for Automated Medical Coding

Description-based Label Attention Classifier for Explainable ICD-9 Classification

Read, Attend, and Code: Pushing the Limits of Medical Codes Prediction from Clinical Notes by Machines

Medical code prediction via capsule networks and ICD knowledge

Towards BERT-based Automatic ICD Coding: Limitations and Opportunities

Does the Magic of BERT Apply to Medical Code Assignment? A Quantitative Study

Limitations of Transformers on Clinical Text Classification

Explainable Automated Coding of Clinical Notes using Hierarchical Label-wise Attention Networks and Label Embedding Initialisation

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Pretrained Transformers for Text Ranking: BERT and Beyond

Explainable Prediction of Medical Codes With Knowledge Graphs

A Comparison of Deep Learning Methods for ICD Coding of Clinical Records

HyperCore: Hyperbolic and Co-graph Representation for Automatic ICD Coding

wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations

Experimental Evaluation and Development of a Silver-Standard for the MIMIC-III Clinical Coding Dataset

BERT-XML: Large Scale Automated ICD Coding Using BERT Pretraining

Inferring Degrees from Incomplete Networks and Nonlinear Dynamics

ICD Coding from Clinical Text Using Multi-Filter Residual Convolutional Neural Network

Macro F1 and Macro F1

EHR Coding with Multi-scale Feature Attention and Structured Knowledge Graph Propagation

An Information Retrieval Approach to ICD-10 Classification

Improving Medical Code Prediction from Clinical Text via Incorporating Online Knowledge Sources

Administrative Costs Associated With Physician Billing and Insurance-Related Activities at an Academic Health Care System

Explainable Prediction of Medical Codes from Clinical Text

Decoupled Weight Decay Regularization

MIMIC-III, a freely accessible critical care database

An empirical evaluation of supervised learning approaches in assigning diagnosis codes to electronic medical records

ICD Code Retrieval: Novel Approach for Assisted Disease Classification

Adam: A Method for Stochastic Optimization

An audit of the nature and impact of clinical coding subjectivity variability and error in otolaryngology

Systematic review of discharge coding accuracy.

Open-access MIMIC-II database for intensive care research

On the Stratification of Multi-label Data

A systematic literature review of automated clinical coding and classification systems

Medical coding classification by leveraging inter-code relationships

Measuring diagnoses: ICD code accuracy.

Physician and coding errors in patient records.

Effective Convolutional Attention Network for Multi-label Clinical Document Classification

Automatic ICD Coding via Interactive Shared Representation Networks with Self-distillation Mechanism

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Physionet: components of a new research resource for complex physiologic signals

We find the original split of MIMIC-III to introduce strong biases in results due to missing classes in the test set

Automated Medical Coding on MIMIC-III and MIMIC-IV

Field of Study

Journal Information

Name

Venue Information

Name

Type

URL

Alternate Names