Deep learning in neural networks: An overview (2014-04-30T00:00:00.000000Z)

TL;DR

This historical survey compactly summarizes relevant work, much of it from the previous millennium, review deep supervised learning, unsupervised learning, reinforcement learning & evolutionary computation, and indirect search for short programs encoding deep and large networks.

Authors

J. Schmidhuber

5 papers

TL;DR

Authors

References995 items

Perceptrons

Dynamic Programming

Reinforcement learning

Unsupervised Learning

Inductive Learning Algorithms for Complex Systems Modeling

The Neural Network Pushdown Automaton: Model, Stack and Learning Simulations

What Is Machine Learning

Cartesian Genetic Programming

Advances in Neural Information Processing Systems (NIPS)

Deep Learning for Real-Time Atari Game Play Using Offline Monte-Carlo Tree Search Planning

Going deeper with convolutions

Deep Learning Based Imaging Data Completion for Improved Brain Disease Diagnosis

Sequence to Sequence Learning with Neural Networks

A million spiking-neuron integrated circuit with a scalable communication network and interface

Leveraging Hierarchical Parametric Networks for Skeletal Joints Based Action Segmentation and Recognition

Large-Scale Video Classification with Convolutional Neural Networks

Automatic Feature Learning for Robust Shadow Detection

Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks

Towards End-To-End Speech Recognition with Recurrent Neural Networks

Deep Learning: Methods and Applications

Variational inference of latent state sequences using Recurrent Networks

Multi-resolution linear prediction based features for audio onset detection with bidirectional LSTM neural networks

Social signal classification using deep blstm recurrent neural networks

The dropout learning algorithm

Reinforcement Learning

The A2iA Arabic Handwritten Text Recognition System at the Open HaRT2013 Evaluation

Stochastic variational learning in recurrent spiking networks

Neural network language models for off-line handwriting recognition

NeuCube: A spiking neural network architecture for mapping, learning and understanding of spatio-temporal brain data

CNN Features Off-the-Shelf: An Astounding Baseline for Recognition

Learning Deep and Wide: A Spectral Method for Learning Deep Networks

A Clockwork RNN

Minitaur, an Event-Driven FPGA-Based Spiking Network Accelerator

OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks

An Empirical Investigation of Catastrophic Forgeting in Gradient-Based Neural Networks

How to Construct Deep Recurrent Neural Networks

Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks

Efficient Visual Coding: From Retina To V2

My First Deep Learning System of 1991 + Deep Learning Timeline 1962-2013

Playing Atari with Deep Reinforcement Learning

Adaptive dropout for training deep neural networks

Deep Neural Networks for Object Detection

Hierarchical Modular Optimization of Convolutional Networks Achieves Representations Similar to Macaque IT and Human Ventral Stream

Compete to Compute

Visualizing and Understanding Convolutional Networks

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

Dropout Improves Recurrent Neural Networks for Handwriting Recognition

Event-driven contrastive divergence for spiking neuromorphic systems

On Fast Dropout and its Applicability to Recurrent Networks

Real-time classification and sensor fusion with a spiking deep belief network

DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition

Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks

Multi-Column Deep Neural Networks for Offline Handwritten Chinese Character Classification

High-Performance OCR for Printed English and Fraktur Using LSTM Networks

ICDAR 2013 Chinese Handwriting Recognition Competition

Deep Hierarchies in the Primate Visual Cortex: What Can We Learn for Computer Vision?

Learning Hierarchical Features for Scene Labeling

Emergence of Dynamic Memory Traces in Cortical Microcircuit Models through STDP

Evolving large-scale neural networks for vision-based reinforcement learning

Critical factors in the performance of hyperNEAT

Cartesian genetic programming encoded artificial neural networks: a comparison using three benchmarks

Deep Architectures and Deep Learning in Chemoinformatics: The Prediction of Aqueous Solubility for Drug-Like Molecules

Deep learning with COTS HPC systems

Fast dropout training

Matching Recall and Storage in Sequence Learning with Spiking Neural Networks

An intrinsic value system for developing multiple invariant representations with incremental slowness learning

Improving deep neural networks for LVCSR using rectified linear units and dropout

Real-life voice activity detection with LSTM Recurrent Neural Networks and an application to Hollywood movies

How to Build a Brain: A Neural Architecture for Biological Cognition

Simulated Car Racing Championship: Competition Software Manual

Training multi-layered neural network neocognitron

Bayesian Computation Emerges in Generic Cortical Microcircuits through Spike-Timing-Dependent Plasticity

Recurrent Processing during Object Recognition

Intrinsically Motivated Learning in Natural and Artificial Systems

Speech recognition with deep recurrent neural networks

Enhanced Gradient for Training Restricted Boltzmann Machines

Maxout Networks