Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing

Published in

IEEE Signal Processing Magazine(2019)

External Links:

Generate Graph DownloadPDF

TL;DR

The increasing popularity of unrolled deep networks is due, in part, to their potential in developing efficient, high-performance (yet interpretable) network architectures from reasonably sized training sets.

Abstract

Deep neural networks provide unprecedented performance gains in many real-world problems in signal and image processing. Despite these gains, the future development and practical deployment of deep networks are hindered by their black-box nature, i.e., a lack of interpretability and the need for very large training sets. An emerging technique called algorithm unrolling, or unfolding, offers promise in eliminating these issues by providing a concrete and systematic connection between iterative algorithms that are widely used in signal processing and deep neural networks. Unrolling methods were first proposed to develop fast neural network approximations for sparse coding. More recently, this direction has attracted enormous attention, and it is rapidly growing in both theoretic investigations and practical applications. The increasing popularity of unrolled deep networks is due, in part, to their potential in developing efficient, high-performance (yet interpretable) network architectures from reasonably sized training sets.

Authors

V. Monga

6 papers

Yuelong Li

1 papers

Yonina C. Eldar

4 papers

References110 items

Compressed Sensing: Theory and Applications

Dense Recurrent Neural Networks for Accelerated MRI: History-Cognizant Unrolling of Optimization Algorithms

Learned SPARCOM: Unfolded Deep Super-Resolution Microscopy

ADMM-CSNet: A Deep Learning Approach for Image Compressive Sensing

Data-Driven Symbol Detection Via Model-Based Machine Learning

Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing

Published in

IEEE Signal Processing Magazine(2019)

External Links:

Generate Graph DownloadPDF

TL;DR

Abstract

Authors

V. Monga

6 papers

Yuelong Li

1 papers

Yonina C. Eldar

4 papers

References110 items

Compressed Sensing: Theory and Applications

Dense Recurrent Neural Networks for Accelerated MRI: History-Cognizant Unrolling of Optimization Algorithms

Learned SPARCOM: Unfolded Deep Super-Resolution Microscopy

ADMM-CSNet: A Deep Learning Approach for Image Compressive Sensing

Data-Driven Symbol Detection Via Model-Based Machine Learning

Efficient and Interpretable Deep Blind Image Deblurring Via Algorithm Unrolling

Dense Recurrent Neural Networks for Inverse Problems: History-Cognizant Unrolling of Optimization Algorithms

Sinkhorn AutoEncoders

Deep Learning in Ultrasound Imaging

ViterbiNet: Symbol Detection Using a Deep Learning Based Viterbi Algorithm

Deep Unfolding for Communications Systems: A Survey and Some New Directions

Plug-and-Play Methods Provably Converge with Properly Trained Denoisers

Unrolled Projected Gradient Descent for Multi-spectral Image Fusion

Adaptive Transform Domain Image Super-Resolution via Orthogonally Regularized Deep Networks

Functional Variational Bayesian Neural Networks

MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentation

An Algorithm Unrolling Approach to Deep Image Deblurring

Deep Algorithm Unrolling for Blind Image Deblurring

PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network

Real-Time Power System State Estimation and Forecasting via Deep Unrolled Neural Networks

Deep Unfolded Robust PCA with Application to Clutter Suppression in Ultrasound

Domain Knowledge Driven Deep Unrolling for Rain Removal from Single Image

ALISTA: Analytic Weights Are As Good As Learned Weights in LISTA

GAN Dissection: Visualizing and Understanding Generative Adversarial Networks

Theoretical Linear Convergence of Unfolded ISTA and its Practical Weights and Thresholds

Sparsity-based super-resolution microscopy from correlation information.

Neural Ordinary Differential Equations

On Multi-Layer Basis Pursuit, Efficient Algorithms and Convolutional Neural Networks

Dynamically Unfolding Recurrent Restorer: A Moving Endpoint Control Method for Image Restoration

Procedural Modeling of a Building from a Single Image

End-to-End Speech Separation with Unfolded Iterative Phase Reconstruction

Scale-Recurrent Network for Deep Image Deblurring

Using Deep Neural Networks for Inverse Problems in Imaging: Beyond Analytical Methods

FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors

DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks

Interpreting Deep Visual Representations via Network Dissection

End-to-End Lung Nodule Detection in Computed Tomography

Wasserstein Auto-Encoders

PDE-Net: Learning PDEs from Data

Maximum Principle Based Algorithms for Deep Learning

CNN-Based Projected Gradient Descent for Consistent CT Image Reconstruction

Learned Primal-Dual Reconstruction

Wasserstein Generative Adversarial Networks

Deep MIMO detection

Learning Generative Models with Sinkhorn Divergences

Unrolled Optimization with Deep Priors

Learned D-AMP: Principled Neural Network based Compressive Image Recovery

Learning Proximal Operators: Using Denoising Networks for Regularizing Inverse Imaging Problems

Neural Expectation Maximization

Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring

Deep Convolutional Neural Network for Inverse Problems in Imaging

Unrolled Generative Adversarial Networks

Convolutional Neural Networks Analyzed via Convolutional Sparse Coding

ReconNet: Non-Iterative Reconstruction of Images from Compressively Sensed Measurements

Deep Learning Markov Random Field for Semantic Segmentation

A Clearer Picture of Total Variation Blind Deconvolution

Maximal Sparsity with Deep Networks?

Deep Residual Learning for Image Recognition

Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration

Deep Networks for Image Super-Resolution with Sparse Prior

Variational Inference with Normalizing Flows

U-Net: Convolutional Networks for Biomedical Image Segmentation

Conditional Random Fields as Recurrent Neural Networks

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

Image Super-Resolution Using Deep Convolutional Networks

Adam: A Method for Stochastic Optimization

Fully convolutional networks for semantic segmentation

A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution

Deep Unfolding: Model-Based Inspiration of Novel Deep Architectures

Learning to Deblur

From Denoising to Compressed Sensing

Auto-Encoding Variational Bayes

Gradient methods for minimizing composite functions

Learning Efficient Sparse and Low Rank Models

ImageNet classification with deep convolutional neural networks

On the difficulty of training recurrent neural networks

Generic Methods for Optimization-Based Modeling

Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials

Parameter learning with truncated message-passing

Learning non-local range Markov Random field for image restoration

Deep Sparse Rectifier Neural Networks

Empirical Risk Minimization of Graphical Model Parameters Given Approximate Inference, Decoding, and Model Structure

Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers

Image Super-Resolution Via Sparse Representation

Task-Driven Dictionary Learning

Learning Fast Approximations of Sparse Coding

Understanding the difficulty of training deep feedforward neural networks

Iterative Phase Estimation for the Synthesis of Separated Sources From Single-Channel Mixtures

Message-passing algorithms for compressed sensing

Coordinate descent optimization for l 1 minimization with application to compressed sensing; a greedy algorithm

ImageNet: A large-scale hierarchical image database

Nonnegative Matrix Factorization with the Itakura-Saito Divergence: With Application to Music Analysis

Sparse coding and NMF

Kalman Filtering and Neural Networks

Learning the parts of objects by non-negative matrix factorization

Neurocontrol of nonlinear dynamical systems with Kalman filter trained recurrent networks

Approximation by superpositions of a sigmoidal function

100

Learning internal representations by error propagation

101

Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position

102

A Technical Report : Prior Information Guided Regularized Deep Learning for Cell Nucleus Detection

103

TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 1 ModDrop

104

Efficient BackProp

105

A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems

106

Gradient-based learning applied to document recognition

107

Training Multilayer Perceptrons with the Extende Kalman Algorithm

108

Receptive fields, binocular interaction and functional architecture in the cat's visual cortex

109

He is currently an applied scientist at Amazon Lab 126. His research interests include computational photography and 3D modeling

110

Authorized licensed use limited to: Weizmann Institute of Science

Field of Study

EngineeringComputer Science

Journal Information

Name

IEEE Signal Processing Magazine

Page

18-44

Volume

Venue Information

Name

IEEE Signal Processing Magazine

Type

journal

URL

http://ieeexplore.ieee.org/servlet/opac?punumber=79

Alternate Names

IEEE Signal Process Mag

TL;DR

Abstract

Authors

References110 items

Compressed Sensing: Theory and Applications

Dense Recurrent Neural Networks for Accelerated MRI: History-Cognizant Unrolling of Optimization Algorithms

Learned SPARCOM: Unfolded Deep Super-Resolution Microscopy

ADMM-CSNet: A Deep Learning Approach for Image Compressive Sensing

Data-Driven Symbol Detection Via Model-Based Machine Learning

TL;DR

Abstract

Authors

References110 items

Compressed Sensing: Theory and Applications

Dense Recurrent Neural Networks for Accelerated MRI: History-Cognizant Unrolling of Optimization Algorithms

Learned SPARCOM: Unfolded Deep Super-Resolution Microscopy

ADMM-CSNet: A Deep Learning Approach for Image Compressive Sensing

Data-Driven Symbol Detection Via Model-Based Machine Learning

Efficient and Interpretable Deep Blind Image Deblurring Via Algorithm Unrolling

Dense Recurrent Neural Networks for Inverse Problems: History-Cognizant Unrolling of Optimization Algorithms

Sinkhorn AutoEncoders

Deep Learning in Ultrasound Imaging

ViterbiNet: Symbol Detection Using a Deep Learning Based Viterbi Algorithm

Deep Unfolding for Communications Systems: A Survey and Some New Directions

Plug-and-Play Methods Provably Converge with Properly Trained Denoisers

Unrolled Projected Gradient Descent for Multi-spectral Image Fusion

Adaptive Transform Domain Image Super-Resolution via Orthogonally Regularized Deep Networks

Functional Variational Bayesian Neural Networks

MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentation

An Algorithm Unrolling Approach to Deep Image Deblurring

Deep Algorithm Unrolling for Blind Image Deblurring

PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network

Real-Time Power System State Estimation and Forecasting via Deep Unrolled Neural Networks

Deep Unfolded Robust PCA with Application to Clutter Suppression in Ultrasound

Domain Knowledge Driven Deep Unrolling for Rain Removal from Single Image

ALISTA: Analytic Weights Are As Good As Learned Weights in LISTA

GAN Dissection: Visualizing and Understanding Generative Adversarial Networks

Theoretical Linear Convergence of Unfolded ISTA and its Practical Weights and Thresholds

Neural Processes

Sparsity-based super-resolution microscopy from correlation information.

Neural Ordinary Differential Equations

On Multi-Layer Basis Pursuit, Efficient Algorithms and Convolutional Neural Networks

Dynamically Unfolding Recurrent Restorer: A Moving Endpoint Control Method for Image Restoration

Procedural Modeling of a Building from a Single Image

End-to-End Speech Separation with Unfolded Iterative Phase Reconstruction

Scale-Recurrent Network for Deep Image Deblurring

Using Deep Neural Networks for Inverse Problems in Imaging: Beyond Analytical Methods

FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors

DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks

Interpreting Deep Visual Representations via Network Dissection

End-to-End Lung Nodule Detection in Computed Tomography

Wasserstein Auto-Encoders

PDE-Net: Learning PDEs from Data

Maximum Principle Based Algorithms for Deep Learning

CNN-Based Projected Gradient Descent for Consistent CT Image Reconstruction

Learned Primal-Dual Reconstruction

Wasserstein Generative Adversarial Networks

Deep MIMO detection

Learning Generative Models with Sinkhorn Divergences

Unrolled Optimization with Deep Priors

Learned D-AMP: Principled Neural Network based Compressive Image Recovery

Learning Proximal Operators: Using Denoising Networks for Regularizing Inverse Imaging Problems

Neural Expectation Maximization

Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring

Deep Convolutional Neural Network for Inverse Problems in Imaging

Unrolled Generative Adversarial Networks

Convolutional Neural Networks Analyzed via Convolutional Sparse Coding

ReconNet: Non-Iterative Reconstruction of Images from Compressively Sensed Measurements

Deep Learning Markov Random Field for Semantic Segmentation

A Clearer Picture of Total Variation Blind Deconvolution

Maximal Sparsity with Deep Networks?

Deep Residual Learning for Image Recognition

Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration

Deep Networks for Image Super-Resolution with Sparse Prior

Variational Inference with Normalizing Flows

U-Net: Convolutional Networks for Biomedical Image Segmentation

Conditional Random Fields as Recurrent Neural Networks

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

Image Super-Resolution Using Deep Convolutional Networks