Learning long-range spatial dependencies with horizontal gated-recurrent units (2018-05-21T00:00:00.000000Z)

TL;DR

This work introduces the horizontal gated-recurrent unit (hGRU) to learn intrinsic horizontal connections -- both within and across feature columns, and demonstrates that a single hGRU layer matches or outperforms all tested feedforward hierarchical baselines including state-of-the-art architectures which have orders of magnitude more free parameters.

Abstract

Progress in deep learning has spawned great successes in many engineering applications. As a prime example, convolutional neural networks, a type of feedforward neural networks, are now approaching -- and sometimes even surpassing -- human accuracy on a variety of visual recognition tasks. Here, however, we show that these neural networks and their recent extensions struggle in recognition tasks where co-dependent visual features must be detected over long spatial ranges. We introduce the horizontal gated-recurrent unit (hGRU) to learn intrinsic horizontal connections -- both within and across feature columns. We demonstrate that a single hGRU layer matches or outperforms all tested feedforward hierarchical baselines including state-of-the-art architectures which have orders of magnitude more free parameters. We further discuss the biological plausibility of the hGRU in comparison to anatomical data from the visual cortex as well as human behavioral data on a classic contour detection task.

Authors

Thomas Serre

3 papers

Drew Linsley

1 papers

Junkyung Kim

1 papers

TL;DR

Abstract

Authors

References77 items

Complementary Surrounds Explain Diverse Contextual Phenomena Across Visual Modalities

Task-Driven Convolutional Recurrent Models of the Visual System

Not-So-CLEVR: learning same–different relations strains feedforward neural networks

Can recurrent neural networks warp time?

Not-So-CLEVR: Visual Relations Strain Feedforward Neural Networks

Deep Crisp Boundaries: From Boundaries to Higher-Level Tasks

A generative vision model that trains with high data efficiency and breaks text-based CAPTCHAs

Non-local Neural Networks

Effective Use of Dilated Convolutions for Segmenting Small Object Instances in Remote Sensing Imagery

Dilated Deep Residual Network for Image Denoising

Deep Crisp Boundaries

Do Deep Neural Networks Suffer from Crowding?

Recurrent Convolutional Neural Networks: A Better Model of Biological Object Recognition

Feedback Networks

Richer Convolutional Features for Edge Detection

Convolutional gated recurrent networks for video segmentation

Learning Long-Term Dependencies for Action Recognition with a Biologically-Inspired Deep Network

Opponent surrounds explain diversity of contextual phenomena across visual modalities

Convolutional Oriented Boundaries: From Image Segmentation to High-Level Tasks

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning

Bridging the Gaps Between Residual Learning, Recurrent Neural Networks and Visual Cortex

Recurrent Batch Normalization

Identity Mappings in Deep Residual Networks

Pixel Recurrent Neural Networks

Inside-Outside Net: Detecting Objects in Context with Skip Pooling and Recurrent Neural Networks

Unsupervised Learning by Program Synthesis

Multi-Scale Context Aggregation by Dilated Convolutions

Pushing the Boundaries of Boundary Detection using Deep Learning

Delving Deeper into Convolutional Networks for Learning Video Representations

Deeply-Recursive Convolutional Network for Image Super-Resolution

SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation

Generative Image Modeling Using Spatial LSTMs

Modeling local and global deformations in Deep Learning: Epitomic convolution, Multiple Instance Learning, and sliding window detection

Recurrent convolutional neural network for object recognition

U-Net: Convolutional Networks for Biomedical Image Segmentation

Highway Networks

ReNet: A Recurrent Neural Network Based Alternative to Convolutional Networks

Holistically-Nested Edge Detection

Conditional Random Fields as Recurrent Neural Networks

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

The Stabilized Supralinear Network: A Unifying Circuit Motif Underlying Multi-Input Integration in Sensory Cortex

Adam: A Method for Stochastic Optimization

Deep neural networks are easily fooled: High confidence predictions for unrecognizable images

Fully convolutional networks for semantic segmentation

Deeply-Supervised Nets

Very Deep Convolutional Networks for Large-Scale Image Recognition

Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

Stereo, Shading, and Surfaces: Curvature Constraints Couple Neural Computations

Intriguing properties of neural networks

Top-down influences on visual processing

Different Orientation Tuning of Near- and Far-Surround Suppression in Macaque Primary Visual Cortex Mirrors Their Tuning in Human Perception

Strong Recurrent Networks Compute the Orientation Tuning of Surround Modulation in the Primate Primary Visual Cortex

Neural circuit models for computations in early visual cortex

Incremental grouping of image elements in vision

Contour Detection and Hierarchical Image Segmentation

Parallel and serial grouping of image elements in visual perception.

Understanding the difficulty of training deep feedforward neural networks

Surround suppression of V1 neurons mediates orientation-based representation of high-order visual features.

Learning to Link Visual Contours

PsychoPy—Psychophysics software in Python

Multi-dimensional Recurrent Neural Networks

Contour Saliency in Primary Visual Cortex

Learning to detect natural image boundaries using local brightness, color, and texture cues

Geometrical Computations Explain Projection Patterns of Long-Range Horizontal Connections in Visual Cortex

The “silent” surround of V1 receptive fields: theory and experiments

Lateral Connectivity and Contextual Interactions in Macaque Primary Visual Cortex

Global contour saliency and local colinear interactions.

Long Short-Term Memory

The role of edges and line-ends in illusory contour formation

Contour integration by the human visual system: Evidence for a local “association field”

Neural dynamics of perceptual grouping: Textures, boundaries, and emergent segmentations

Intrinsic laminar lattice connections in primate visual cortex

2015, February). Deeply-Supervised nets. In Artificial intelligence and statistics (pp. 562–570)

Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks