Assessment of data consistency through cascades of independently recurrent inference machines for fast and robust accelerated MRI reconstruction (2021-11-30T00:00:00.000000Z)

TL;DR

The CIRIM showed highly promising generalization capabilities maintaining a very fair trade-off between reconstructed image quality and fast reconstruction times, which is crucial in the clinical workflow.

Abstract

Objective. Machine Learning methods can learn how to reconstruct magnetic resonance images (MRI) and thereby accelerate acquisition, which is of paramount importance to the clinical workflow. Physics-informed networks incorporate the forward model of accelerated MRI reconstruction in the learning process. With increasing network complexity, robustness is not ensured when reconstructing data unseen during training. We aim to embed data consistency (DC) in deep networks while balancing the degree of network complexity. While doing so, we will assess whether either explicit or implicit enforcement of DC in varying network architectures is preferred to optimize performance. Approach. We propose a scheme called Cascades of Independently Recurrent Inference Machines (CIRIM) to assess DC through unrolled optimization. Herein we assess DC both implicitly by gradient descent and explicitly by a designed term. Extensive comparison of the CIRIM to compressed sensing as well as other Machine Learning methods is performed: the End-to-End Variational Network (E2EVN), CascadeNet, KIKINet, LPDNet, RIM, IRIM, and UNet. Models were trained and evaluated on T1-weighted and FLAIR contrast brain data, and T2-weighted knee data. Both 1D and 2D undersampling patterns were evaluated. Robustness was tested by reconstructing 7.5× prospectively undersampled 3D FLAIR MRI data of multiple sclerosis (MS) patients with white matter lesions. Main results. The CIRIM performed best when implicitly enforcing DC, while the E2EVN required an explicit DC formulation. Through its cascades, the CIRIM was able to score higher on structural similarity and PSNR compared to other methods, in particular under heterogeneous imaging conditions. In reconstructing MS patient data, prospectively acquired with a sampling pattern unseen during model training, the CIRIM maintained lesion contrast while efficiently denoising the images. Significance. The CIRIM showed highly promising generalization capabilities maintaining a very fair trade-off between reconstructed image quality and fast reconstruction times, which is crucial in the clinical workflow.

References86 items

Partial Fourier reconstruction of complex MR images using complex‐valued convolutional neural networks

fastMRI+: Clinical Pathology Annotations for Knee and Brain Fully Sampled Multi-Coil MRI Data

Spatio-temporal multi-task network cascade for accurate assessment of cardiac CT perfusion

Multi-view convolutional neural networks for automated ocular structure and tumor segmentation in retinoblastoma

Systematic evaluation of iterative deep neural networks for fast parallel MRI reconstruction with sensitivity‐weighted coil combination

Joint Deep Model-based MR Image and Coil Sensitivity Reconstruction Network (Joint-ICNet) for Fast MRI

Unsupervised MRI Reconstruction via Zero-Shot Learned Adversarial Transformers

Projection-Based cascaded U-Net model for MR image reconstruction

Results of the 2020 fastMRI Challenge for Machine Learning MR Image Reconstruction

Analysis of deep complex‐valued convolutional neural networks for MRI reconstruction and phase‐focused applications

Memory-efficient Learning for High-Dimensional MRI Reconstruction

State-of-the-Art Machine Learning MRI Reconstruction in 2020: Results of the Second fastMRI Challenge

A deep unrolling network inspired by total variation for compressed sensing MRI

XPDNet for MRI Reconstruction: an Application to the fastMRI 2020 Brain Challenge

Multi-channel MR Reconstruction (MC-MRRec) Challenge - Comparing Accelerated MR Reconstruction Models and Assessing Their Genereralizability to Datasets Collected with Different Coils

Prior-Guided Image Reconstruction for Accelerated Multi-Contrast MRI via Generative Adversarial Networks

Unsupervised MRI Reconstruction with Generative Adversarial Networks

An Adaptive Intelligence Algorithm for Undersampled Knee MRI Reconstruction

End-to-End Variational Networks for Accelerated MRI Reconstruction

Benchmarking Deep Nets MRI Reconstruction Models on the Fastmri Publicly Available Dataset

Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge

Σ-net: Ensembled Iterative Deep Neural Networks for Accelerated Parallel MR Image Reconstruction

PyTorch: An Imperative Style, High-Performance Deep Learning Library

Pyramid Convolutional RNN for MRI Image Reconstruction

Dual-domain Cascade of U-nets for Multi-channel Magnetic Resonance Image Reconstruction

GrappaNet: Combining Parallel Imaging With Deep Learning for Multi-Coil MRI Reconstruction

i-RIM applied to the fastMRI challenge

Optimal Transport Driven CycleGAN for Unsupervised Learning in Inverse Problems

VS-Net: Variable splitting network for accelerated parallel MRI reconstruction

DeepcomplexMRI: Exploiting deep residual network for fast parallel MR imaging with complex convolution

LORAKI: Autocalibrated Recurrent Neural Networks for Autoregressive MRI Reconstruction in k-Space

Recurrent inference machines for reconstructing heterogeneous MRI data

A Hybrid, Dual Domain, Cascade of Convolutional Neural Networks for Magnetic Resonance Image Reconstruction

fastMRI: An Open Dataset and Benchmarks for Accelerated MRI

MRI Reconstruction Via Cascaded Channel-Wise Attention Network

Scan‐specific robust artificial‐neural‐networks for k‐space interpolation (RAKI) reconstruction: Database‐free deep learning for fast imaging

DAGAN: Deep De-Aliasing Generative Adversarial Networks for Fast Compressed Sensing MRI Reconstruction

KIKI‐net: cross‐domain convolutional neural networks for reconstructing undersampled magnetic resonance images

Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN

MoDL: Model-Based Deep Learning Architecture for Inverse Problems

Convolutional Recurrent Neural Networks for Dynamic MR Image Reconstruction

A Transfer‐Learning Approach for Accelerated MRI Using Deep Neural Networks

Compressed Sensing MRI Reconstruction Using a Generative Adversarial Network With a Cyclic Loss

Learned Primal-Dual Reconstruction

Recurrent Inference Machines for Solving Inverse Problems

ADMM-Net: A Deep Learning Approach for Compressive Sensing MRI

Image reconstruction by domain-transform manifold learning

A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction

Learning a variational network for reconstruction of accelerated MRI data

Is a Complex-Valued Stepsize Advantageous in Complex-Valued Gradient Learning Algorithms?

Learning to learn by gradient descent by gradient descent

Learning Representations Using Complex-Valued Nets

U-Net: Convolutional Networks for Biomedical Image Segmentation

On learning optimized reaction diffusion processes for effective image restoration

Adam: A Method for Stochastic Optimization

Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

ESPIRiT—an eigenvalue approach to autocalibrating parallel MRI: Where SENSE meets GRAPPA

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

On the difficulty of training recurrent neural networks

Wirtinger Calculus Based Gradient Descent and Levenberg-Marquardt Learning Algorithms in Complex-Valued Neural Networks

Slow and fast scales for superprocess limits of age-structured populations

Compressed Sensing MRI

Sparse MRI: The application of compressed sensing for rapid MR imaging

Image quality assessment: from error visibility to structural similarity

On the Surprising Behavior of Distance Metrics in High Dimensional Spaces

SENSE: Sensitivity encoding for fast MRI

Simultaneous acquisition of spatial harmonics (SMASH): Fast imaging with radiofrequency coil arrays

DIRECT: Deep Image REConstruction Toolkit 2022

Evaluation of the Robustness of Learned MR Image Reconstruction to Systematic Deviations Between Training and Test Data for the Models from the fastMRI Challenge

MRI Data Consistency.

andCukur T 2020a Prior-guided image reconstruction for acceleratedmulti-contrast

A and SodicksonDK2020bGrappaNet: combining parallel imagingwith deep learning

CaanMWAandWellingM2019Recurrent inferencemachines for reconstructing

RMandHammernikK 2019Σ-net: Ensembled Iterative DeepNeuralNetworks for Accelerated

andÖktemO2018 Learned primal-dual reconstruction IEEETrans.Med. Imaging 37 1322–32 AggarwalHK,ManiMP and JacobM2019MoDL:model-based deep learning architecture for inverse problems IEEETrans.Med

PyTorch Lightning The lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate

LeeH J andHwangD2018KIKI-net: cross-domain convolutional neural networks for reconstructing

Statistical Parametric Mapping The Analysis Of Functional Brain Images

Berkeley Advanced Reconstruction Toolbox

Creation of fully sampled MR data repository for compressed sensing of the knee SMRT Conf

Compressed Sensing

Linear Inverse Problems

A Threshold Selection Method from Gray-Level Histograms

KIKINet (third row-first), and the CascadeNet (third row-third) enforced DC explicitly by a formulated DC term

The CIRIM 5C (first row-fourth), the RIM (second row-second), and the IRIM (second row-fourth) enforced Data Consistency (DC) implicitly by gradient descent

Submitted to Magnetic Resonance in Medicine Scan Specific Artifact Reduction in K-space (SPARK) Neural Networks Synergize with Physics-based Reconstruction to Accelerate MRI