Learning curves for drug response prediction in cancer cell lines (2020-11-25T00:00:00.000000Z)

TL;DR

Empirical learning curves are utilized for evaluating and comparing the data scaling properties of two neural networks and two gradient boosting decision tree models trained on four cell line drug screening datasets, demonstrating the benefit of using learning curves to evaluate prediction models and providing a broader perspective on the overall data scaling characteristics.

Abstract

Motivated by the size and availability of cell line drug sensitivity data, researchers have been developing machine learning (ML) models for predicting drug response to advance cancer treatment. As drug sensitivity studies continue generating drug response data, a common question is whether the generalization performance of existing prediction models can be further improved with more training data. We utilize empirical learning curves for evaluating and comparing the data scaling properties of two neural networks (NNs) and two gradient boosting decision tree (GBDT) models trained on four cell line drug screening datasets. The learning curves are accurately fitted to a power law model, providing a framework for assessing the data scaling behavior of these models. The curves demonstrate that no single model dominates in terms of prediction performance across all datasets and training sizes, thus suggesting that the actual shape of these curves depends on the unique pair of an ML model and a dataset. The multi-input NN (mNN), in which gene expressions of cancer cells and molecular drug descriptors are input into separate subnetworks, outperforms a single-input NN (sNN), where the cell and drug features are concatenated for the input layer. In contrast, a GBDT with hyperparameter tuning exhibits superior performance as compared with both NNs at the lower range of training set sizes for two of the tested datasets, whereas the mNN consistently performs better at the higher range of training sizes. Moreover, the trajectory of the curves suggests that increasing the sample size is expected to further improve prediction scores of both NNs. These observations demonstrate the benefit of using learning curves to evaluate prediction models, providing a broader perspective on the overall data scaling characteristics. A fitted power law learning curve provides a forward-looking metric for analyzing prediction performance and can serve as a co-design tool to guide experimental biologists and computational scientists in the design of future experiments in prospective research studies.

References44 items

Enhanced Co-Expression Extrapolation (COXEN) Gene Selection Method for Building Anti-Cancer Drug Response Prediction Models

Machine learning approaches to drug response prediction: challenges and recent progress

Ensemble transfer learning for the prediction of anti-cancer drug response

Representation of features as images with neighborhood dependencies for compatibility with convolutional neural networks

EfficientDet: Scalable and Efficient Object Detection

A Constructive Prediction of the Generalization Error Across Scales

Machine learning and data mining frameworks for predicting drug response in cancer: An overview and a novel in silico screening process based on association rule mining.

Towards Explainable Anticancer Compound Sensitivity Prediction via Multimodal Attention-based Convolutional Encoders

Dr.VAE: improving drug response prediction via modeling of drug perturbation effects

MOLI: multi-omics late integration with deep neural networks for drug response prediction

Predicting tumor cell line response to drug pairs with deep learning

Application of transfer learning for cancer drug sensitivity prediction

KekuleScope: prediction of cancer cell line sensitivity and compound potency using convolutional neural networks trained on compound images

Genetic and transcriptional evolution alters cancer cell line drug response

Predicting drug response of tumors from integrated genomic profiles by deep neural networks

Mordred: a molecular descriptor calculator

Universal Language Model Fine-tuning for Text Classification

LightGBM: A Highly Efficient Gradient Boosting Decision Tree

Deep Learning Scaling is Predictable, Empirically

Revisiting Unreasonable Effectiveness of Data in Deep Learning Era

A Next Generation Connectivity Map: L1000 Platform and the First 1,000,000 Profiles

Algorithms for Drug Sensitivity Prediction

Harnessing Connectivity in a Large-Scale Small-Molecule Sensitivity Dataset.

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Adam: A Method for Stochastic Optimization

A community effort to assess and improve drug sensitivity prediction algorithms

Systematic Assessment of Analytical Methods for Drug Sensitivity Prediction from Cancer Cell Line Data

The clinical relevance of cancer cell lines.

Genomics of Drug Sensitivity in Cancer (GDSC): a resource for therapeutic biomarker discovery in cancer cells

Predicting sample size required for classification performance

Random Search for Hyper-Parameter Optimization

Learning curve models and applications: Literature review and research directions

Cell line-based platforms to evaluate the therapeutic efficacy of candidate anticancer agents

Estimating Dataset Size Requirements for Classifying DNA Microarray Data

Learning Curves: Asymptotic Values and Rate of Convergence

The National Cancer Institute: cancer drug discovery and development program.

Four Types of Learning Curves

Predicting accuracy on large datasets from smaller pilot data

Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

Universal language model fine-tuning for text classification. In: Proceedings of the 56th annual meeting of the association for computational linguistics (volume 1: long papers)

Keras. https:// keras

Dropout: a simple way to prevent neural networks from overfitting

Predicting and Optimizing Classifier Utility with the Power Law

Rigorous Learning Curve Bounds from Statistical Mechanics