Leakage and the reproducibility crisis in machine-learning-based science

Published in

Patterns(2023)

External Links:

Generate Graph DownloadPDF

TL;DR

A survey of literature in fields that have adopted ML methods finds 17 fields where leakage has been found, collectively affecting 294 papers and, in some cases, leading to wildly overoptimistic conclusions.

Authors

Sayash Kapoor

3 papers

Arvind Narayanan

2 papers

References96 items

The Worst of Both Worlds: A Comparative Analysis of Errors in Learning from Data in Psychology and Machine Learning

Assessing Methods and Tools to Improve Reporting, Increase Transparency, and Reduce Failures in Machine Learning Applications in Health Care.

Genomic Machine Learning Meta-regression: Insights on Associations of Study Features With Reported Model Performance

Navigating the pitfalls of applying machine learning in genomics

AI and the Everything in the Whole Wide World Benchmark

Leakage and the reproducibility crisis in machine-learning-based science

Published in

Patterns(2023)

External Links:

Generate Graph DownloadPDF

TL;DR

Authors

Sayash Kapoor

3 papers

Arvind Narayanan

2 papers

References96 items

The Worst of Both Worlds: A Comparative Analysis of Errors in Learning from Data in Psychology and Machine Learning

Assessing Methods and Tools to Improve Reporting, Increase Transparency, and Reduce Failures in Machine Learning Applications in Health Care.

Genomic Machine Learning Meta-regression: Insights on Associations of Study Features With Reported Model Performance

Navigating the pitfalls of applying machine learning in genomics

AI and the Everything in the Whole Wide World Benchmark

Confounds in the Data—Comments on “Decoding Brain Representations by Multimodal Learning of Neural Activity and Visual Features”

Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Development

Avoiding common machine learning pitfalls

An Empirical Study of the Impact of Data Splitting Decisions on the Performance of AIOps Solutions

Integrating explanation and prediction in computational social science

Scientific Credibility of Machine Translation Research: A Meta-Evaluation of 769 Papers

A computational reproducibility study of PLOS ONE articles featuring longitudinal data analyses

Nonreplicable publications are cited more than replicable ones

Perspectives on Machine Learning from Psychology's Reproducibility Crisis

Inflated prediction accuracy of neuropsychiatric biomarkers caused by data leakage in feature selection

Reproducibility in machine learning for health research: Still a ways to go

Data Leakage in Health Outcomes Prediction With Machine Learning. Comment on “Prediction of Incident Hypertension Within the Next Year: Prospective Study Using Statewide Electronic Health Records and Machine Learning”

Data and its (dis)contents: A survey of dataset development and use in machine learning research

The fundamental principles of reproducibility

The Perils and Pitfalls of Block Design for EEG Classification Experiments

Putting Psychology to the Test: Rethinking Model Evaluation Through Benchmarking and Prediction

Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans

The Politics and Science of Prevision

Forecasting civil war and political violence

Training machine learning models on patient level data segregation is crucial in practical clinical applications

Forecasting Civil Wars: Theory and Structure in an Age of “Big Data” and Machine Learning

Expanding the Scope of Reproducibility Research Through Data Analysis Replications

Shortcut learning in deep neural networks

Measuring the predictability of life outcomes with a scientific mass collaboration

Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)

Overinterpretation reveals image classification model pathologies

Checklist for Artificial Intelligence in Medical Imaging (CLAIM): A Guide for Authors and Reviewers.

Questionable Classification Accuracy Reported in “Designing a Sum of Squared Correlations Framework for Enhancing SSVEP-Based BCIs”

A Hierarchy of Limitations in Machine Learning

Overly optimistic prediction results on imbalanced data: a case study of flaws and benefits when applying over-sampling

What is Your Estimand? Defining the Target Quantity Connects Statistical Evidence to Theory

Establishment of Best Practices for Evidence for Prediction: A Review.

Computational Reproducibility via Containers in Psychology

No coding required: Companies make it easier than ever for scientists to use artificial intelligence

We Need to Talk about Standard Splits

The trim-and-fill method for publication bias: practical guidelines and recommendations based on a large database of meta-analyses

Machine Learning Methods That Economists Should Know About

Improving Supreme Court Forecasting Using Boosted Decision Trees

Tractography and machine learning: Current state and open challenges

Do ImageNet Classifiers Generalize to ImageNet?

A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models.

Successes and Struggles with Computational Reproducibility: Lessons from the Fragile Families Challenge

Comparing Random Forest with Logistic Regression for Predicting Class-Imbalanced Civil War Onset Data: A Comment

Oy Vey! A Comment on "Machine Learning of Toxicological Big Data Enables Read-Across Structure Activity Relationships Outperforming Animal Test Reproducibility".

Validating Hyperspectral Image Segmentation

Be careful of when: an empirical study on time-related misuse of issue tracking data

Decoding Brain Representations by Multimodal Learning of Neural Activity and Visual Features

ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness

Easing the burden of code review

Dimensions: Building Context for Search and Evaluation

Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study

Datasheets for datasets

An empirical analysis of journal policy effectiveness for computational reproducibility

Meta-analysis and the science of research synthesis

Supervised Machine Learning for Population Genetics: A New Paradigm

The accuracy, fairness, and limits of predicting recidivism

50 Years of Data Science

A checklist is associated with increased quality of reporting preclinical biomedical research: A systematic review

Choosing Prediction Over Explanation in Psychology: Lessons From Machine Learning

Data-driven advice for applying machine learning to bioinformatics problems

Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure

Reproducibility of Benchmarked Deep Reinforcement Learning Tasks for Continuous Control

Prediction and explanation in social systems

Applications of machine learning in animal behaviour studies

Machine learning, statistical learning and the future of biological research in psychiatry

The importance of prediction model validation and assessment in obesity and nutrition research

Joint use of over- and under-sampling techniques and cross-validation for the development and assessment of prediction models

Estimating the reproducibility of psychological science

Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

Opinion: Reproducible research can still be wrong: Adopting a prevention approach

Applying Machine Learning to Facilitate Autism Diagnostics: Pitfalls and Promises

ImageNet Large Scale Visual Recognition Challenge

When Optimism Hurts: Inflated Predictions in Psychiatric Neuroimaging

Intriguing properties of neural networks

Reporting and Methods in Clinical Prediction Research: A Systematic Review

Leakage in data mining: formulation, detection, and avoidance

pROC: an open-source package for R and S+ to analyze and compare ROC curves

The perils of policy by p-value: Predicting civil conflicts

Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author)

Block Cross-Validation for Species Distribution Modelling

Computer Vision: Algorithms and Applications, 2nd ed. https://szeliski.org/Book

Journal and Country Rank

Top 10 Ways Your Machine Learning Models May Have Leakage

Comparing Random Forest with Logistic Regression for Predicting Class-Imbalanced Civil War Onset Data

TL;DR

Authors

References96 items

The Worst of Both Worlds: A Comparative Analysis of Errors in Learning from Data in Psychology and Machine Learning

Assessing Methods and Tools to Improve Reporting, Increase Transparency, and Reduce Failures in Machine Learning Applications in Health Care.

Genomic Machine Learning Meta-regression: Insights on Associations of Study Features With Reported Model Performance

Navigating the pitfalls of applying machine learning in genomics

AI and the Everything in the Whole Wide World Benchmark

TL;DR

Authors

References96 items

The Worst of Both Worlds: A Comparative Analysis of Errors in Learning from Data in Psychology and Machine Learning

Assessing Methods and Tools to Improve Reporting, Increase Transparency, and Reduce Failures in Machine Learning Applications in Health Care.

Genomic Machine Learning Meta-regression: Insights on Associations of Study Features With Reported Model Performance

Navigating the pitfalls of applying machine learning in genomics

AI and the Everything in the Whole Wide World Benchmark

Confounds in the Data—Comments on “Decoding Brain Representations by Multimodal Learning of Neural Activity and Visual Features”

Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Development

Avoiding common machine learning pitfalls

An Empirical Study of the Impact of Data Splitting Decisions on the Performance of AIOps Solutions

Integrating explanation and prediction in computational social science

Scientific Credibility of Machine Translation Research: A Meta-Evaluation of 769 Papers

A computational reproducibility study of PLOS ONE articles featuring longitudinal data analyses

Nonreplicable publications are cited more than replicable ones

Perspectives on Machine Learning from Psychology's Reproducibility Crisis

Inflated prediction accuracy of neuropsychiatric biomarkers caused by data leakage in feature selection

Reproducibility in machine learning for health research: Still a ways to go

Data Leakage in Health Outcomes Prediction With Machine Learning. Comment on “Prediction of Incident Hypertension Within the Next Year: Prospective Study Using Statewide Electronic Health Records and Machine Learning”

Data and its (dis)contents: A survey of dataset development and use in machine learning research

The fundamental principles of reproducibility

The Perils and Pitfalls of Block Design for EEG Classification Experiments

Putting Psychology to the Test: Rethinking Model Evaluation Through Benchmarking and Prediction

Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans

The Politics and Science of Prevision

Forecasting civil war and political violence

Training machine learning models on patient level data segregation is crucial in practical clinical applications

Forecasting Civil Wars: Theory and Structure in an Age of “Big Data” and Machine Learning

Expanding the Scope of Reproducibility Research Through Data Analysis Replications

Shortcut learning in deep neural networks

Measuring the predictability of life outcomes with a scientific mass collaboration

Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)

Overinterpretation reveals image classification model pathologies

Checklist for Artificial Intelligence in Medical Imaging (CLAIM): A Guide for Authors and Reviewers.

Questionable Classification Accuracy Reported in “Designing a Sum of Squared Correlations Framework for Enhancing SSVEP-Based BCIs”

A Hierarchy of Limitations in Machine Learning

Overly optimistic prediction results on imbalanced data: a case study of flaws and benefits when applying over-sampling

What is Your Estimand? Defining the Target Quantity Connects Statistical Evidence to Theory

Establishment of Best Practices for Evidence for Prediction: A Review.

Computational Reproducibility via Containers in Psychology

No coding required: Companies make it easier than ever for scientists to use artificial intelligence

We Need to Talk about Standard Splits

The trim-and-fill method for publication bias: practical guidelines and recommendations based on a large database of meta-analyses

Machine Learning Methods That Economists Should Know About

Improving Supreme Court Forecasting Using Boosted Decision Trees

Tractography and machine learning: Current state and open challenges

Do ImageNet Classifiers Generalize to ImageNet?

A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models.

Successes and Struggles with Computational Reproducibility: Lessons from the Fragile Families Challenge

Comparing Random Forest with Logistic Regression for Predicting Class-Imbalanced Civil War Onset Data: A Comment

Oy Vey! A Comment on "Machine Learning of Toxicological Big Data Enables Read-Across Structure Activity Relationships Outperforming Animal Test Reproducibility".

Validating Hyperspectral Image Segmentation

Be careful of when: an empirical study on time-related misuse of issue tracking data

Decoding Brain Representations by Multimodal Learning of Neural Activity and Visual Features

ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness

Easing the burden of code review

Dimensions: Building Context for Search and Evaluation

Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study

Big Data Methods

Datasheets for datasets

An empirical analysis of journal policy effectiveness for computational reproducibility

Meta-analysis and the science of research synthesis

Supervised Machine Learning for Population Genetics: A New Paradigm

The accuracy, fairness, and limits of predicting recidivism

50 Years of Data Science

A checklist is associated with increased quality of reporting preclinical biomedical research: A systematic review

Choosing Prediction Over Explanation in Psychology: Lessons From Machine Learning

Data-driven advice for applying machine learning to bioinformatics problems

Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure

Reproducibility of Benchmarked Deep Reinforcement Learning Tasks for Continuous Control

Do the robot