Slideflow: deep learning for digital histopathology with real-time whole-slide visualization (2023-04-09T00:00:00.000000Z)

TL;DR

A flexible deep learning library for histopathology called Slideflow is developed, a package which supports a broad array of deep learning methods for digital pathology and includes a fast whole-slide interface for deploying trained models.

Abstract

Deep learning methods have emerged as powerful tools for analyzing histopathological images, but current methods are often specialized for specific domains and software environments, and few open-source options exist for deploying models in an interactive interface. Experimenting with different deep learning approaches typically requires switching software libraries and reprocessing data, reducing the feasibility and practicality of experimenting with new architectures. We developed a flexible deep learning library for histopathology called Slideflow, a package which supports a broad array of deep learning methods for digital pathology and includes a fast whole-slide interface for deploying trained models. Slideflow includes unique tools for whole-slide image data processing, efficient stain normalization and augmentation, weakly-supervised whole-slide classification, uncertainty quantification, feature generation, feature space analysis, and explainability. Whole-slide image processing is highly optimized, enabling whole-slide tile extraction at 40x magnification in 2.5 s per slide. The framework-agnostic data processing pipeline enables rapid experimentation with new methods built with either Tensorflow or PyTorch, and the graphical user interface supports real-time visualization of slides, predictions, heatmaps, and feature space characteristics on a variety of hardware devices, including ARM-based devices such as the Raspberry Pi.

Authors

References76 items

Prevalence and predictors of data and code sharing in the medical and health sciences: systematic review with meta-analysis of individual participant data

From patterns to patients: Advances in clinical machine learning for cancer diagnosis, prognosis, and treatment

Abstract P2-11-08: Multimodal Prediction of Breast Cancer Recurrence Assays and Risk of Recurrence

histolab: A Python library for reproducible Digital Pathology preprocessing with automated testing

Deep learning-enabled virtual histological staining of biological samples

Deep learning generates synthetic cancer histology for explainability and education

MONAI: An open-source framework for deep learning in healthcare

RetCCL: Clustering-guided contrastive learning for whole-slide image retrieval

TIAToolbox as an end-to-end library for advanced tissue image analytics

Transformer-based unsupervised contrastive learning for histopathological image classification

Multiple Instance Learning for Digital Pathology: A Review on the State-of-the-Art, Limitations & Future Potential

Multimodal data integration using machine learning improves risk stratification of high-grade serous ovarian cancer

Data augmentation and multimodal learning for predicting drug response in patient-derived xenografts from gene expressions and histology images

Uncertainty-informed deep learning models enable high-confidence predictions for digital histopathology

Building Tools for Machine Learning and Artificial Intelligence in Cancer Research: Best Practices and a Case Study with the PathML Toolkit for Computational Pathology

SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization

Benchmarking artificial intelligence methods for end-to-end computational pathology

Explainable artificial intelligence (XAI) in deep learning-based medical image analysis

The impact of site-specific digital histology signatures on deep learning model accuracy and bias

PathML: A unified framework for whole-slide image analysis with deep learning

Alias-Free Generative Adversarial Networks

TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classication

Deep learning in histopathology: the path to the clinic

Deep learning detects genetic alterations in cancer histology generated by adversarial networks

Second opinion needed: communicating uncertainty in medical machine learning

Deep learning prediction of BRAF-RAS gene expression signature identifies noninvasive follicular thyroid neoplasms with papillary-like nuclear features

A deep learning model to predict RNA-Seq expression of tumours from whole slide images

Clinical-grade Detection of Microsatellite Instability in Colorectal Tumors by Deep Learning.

Novel artificial intelligence system increases the detection of prostate cancer in whole slide images of core needle biopsies

PyHIST: A Histological Image Segmentation Tool

Data-efficient and weakly supervised computational pathology on whole-slide images

fastai: A Layered API for Deep Learning

Cellpose: a generalist algorithm for cellular segmentation

Analyzing and Improving the Image Quality of StyleGAN

Pan-cancer image-based detection of clinically actionable genetic alterations

Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis

Clinical-grade computational pathology using weakly supervised deep learning on whole slide images

Segment Integrated Gradients: Better attributions through regions

Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer

Deep learning with multimodal representation for pancancer prognosis prediction

The need for uncertainty quantification in machine-assisted medical decision making

DeepFocus: Detection of out-of-focus regions in whole slide digital images using deep learning

Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning

Whole-Slide Mitosis Detection in H&E Breast Histology Using PHH3 as a Reference to Train Distilled Stain-Invariant Convolutional Networks

A disciplined approach to neural network hyper-parameters: Part 1 - learning rate, batch size, momentum, and weight decay

Attention-based Deep Multiple Instance Learning

UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction

Axiomatic Attribution for Deep Networks

QuPath: Open source software for digital pathology image analysis

Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization

Structure-Preserving Color Normalization and Sparse Stain Separation for Histological Images

TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems

Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning

Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps

The Cancer Genome Atlas Pan-Cancer analysis project

A method for normalizing histology slides for quantitative analysis

Human Papillomavirus and Head and Neck Squamous Cell Carcinoma: Recent Evidence and Clinical Implications

VIPS - a highly tuned image processing software architecture

Prevalence of histological findings of human papillomavirus (HPV) in oral and oropharyngeal squamous cell carcinoma biopsies: preliminary study

Color Transfer between Images

Lanczos Filtering in One and Two Dimensions

cuCIM - A GPU image I/O and processing library

“Segmentation Models Pytorch,”

“Style Guide for Python Code,”

A Threshold Selection Method from Gray-Level Histograms

Distributed under Creative Commons Cc-by 4.0 Scikit-image: Image Processing in Python

tumor-segmentation-v1 (Revision 666fa9d)

UMAP plot of post-convolutional layer activations, calculated using the final trained model, for all images in the TCGA test set

“Google Python Style Guide.”

Feature space visualization for the tile-based and multiple-instance learning models. (a) UMAP plot of postconvolutional layer activations, calculated using the final trained model