Deep Reinforcement Learning: A Brief Survey (2017-08-19T00:00:00.000000Z)

TL;DR

This survey will cover central algorithms in deep RL, including the deep Q-network (DQN), trust region policy optimization (TRPO), and asynchronous advantage actor critic, and highlight the unique advantages of deep neural networks, focusing on visual understanding via RL.

Abstract

Deep reinforcement learning (DRL) is poised to revolutionize the field of artificial intelligence (AI) and represents a step toward building autonomous systems with a higherlevel understanding of the visual world. Currently, deep learning is enabling reinforcement learning (RL) to scale to problems that were previously intractable, such as learning to play video games directly from pixels. DRL algorithms are also applied to robotics, allowing control policies for robots to be learned directly from camera inputs in the real world. In this survey, we begin with an introduction to the general field of RL, then progress to the main streams of value-based and policy-based methods. Our survey will cover central algorithms in deep RL, including the deep Q-network (DQN), trust region policy optimization (TRPO), and asynchronous advantage actor critic. In parallel, we highlight the unique advantages of deep neural networks, focusing on visual understanding via RL. To conclude, we describe several current areas of research within the field.

Authors

Kai Arulkumaran

1 papers

M. Deisenroth

1 papers

Miles Brundage

3 papers

TL;DR

Abstract

Authors

References183 items

Building Machines that Learn and Think Like People

StarCraft II: A New Challenge for Reinforcement Learning

Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning

Proximal Policy Optimization Algorithms

Learning model-based planning from scratch

Imagination-Augmented Agents for Deep Reinforcement Learning

A Distributional Perspective on Reinforcement Learning

Distral: Robust multitask reinforcement learning

Emergence of Locomotion Behaviours in Rich Environments

Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics

Interpolated Policy Gradient: Merging On-Policy and Off-Policy Gradient Estimation for Deep Reinforcement Learning

Curiosity-Driven Exploration by Self-Supervised Prediction

Discrete Sequential Prediction of Continuous Actions for Deep RL

Equivalence Between Policy Gradients and Soft Q-Learning

The Reactor: A fast and sample-efficient Actor-Critic agent for Reinforcement Learning

The Reactor: A Sample-Efficient Actor-Critic Architecture

Deep Q-learning From Demonstrations

Learning from Demonstrations for Real World Reinforcement Learning

Recurrent Environment Simulators

Multiagent Bidirectionally-Coordinated Nets for Learning to Play StarCraft Combat Games

Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Games

Evolution Strategies as a Scalable Alternative to Reinforcement Learning

Neural Episodic Control

Third-Person Imitation Learning

FeUdal Networks for Hierarchical Reinforcement Learning

Bridging the Gap Between Value and Policy Based Reinforcement Learning

Neural Map: Structured Memory for Deep Reinforcement Learning

Multi-agent Reinforcement Learning in Sequential Social Dilemmas

Deep Reinforcement Learning: An Overview

DeepMind Lab

Model-based Adversarial Imitation Learning

Learning to reinforcement learn

#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning

Learning to Navigate in Complex Environments

PGQ: Combining policy gradient and Q-learning

Reinforcement Learning with Unsupervised Auxiliary Tasks

Neural Architecture Search with Reinforcement Learning

Q-Prop: Sample-Efficient Policy Gradient with An Off-Policy Critic

Learning to Perform Physics Experiments via Deep Reinforcement Learning

RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning

Sample Efficient Actor-Critic with Experience Replay

TorchCraft: a Library for Machine Learning Research on Real-Time Strategy Games

Sim-to-Real Robot Learning from Pixels with Progressive Nets

Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Model

Towards Deep Symbolic Reinforcement Learning

Target-driven visual navigation in indoor scenes using deep reinforcement learning

The Option-Critic Architecture

Episodic Exploration for Deep Deterministic Policies: An Application to StarCraft Micromanagement Tasks

Deep Active Learning for Autonomous Navigation

An Actor-Critic Algorithm for Sequence Prediction

The Malmo Platform for Artificial Intelligence Experimentation

Strategic Attentive Writer for Learning Macro-Actions

Progressive Neural Networks

Generative Adversarial Imitation Learning

Deep Successor Reinforcement Learning

Safe and Efficient Off-Policy Reinforcement Learning

Learning to Optimize

Unifying Count-Based Exploration and Intrinsic Motivation

OpenAI Gym

Control of Memory, Active Perception, and Action in Minecraft

Learning Multiagent Communication with Backpropagation

Learning to Communicate with Deep Multi-Agent Reinforcement Learning

ViZDoom: A Doom-based AI research platform for visual reinforcement learning

VIME: Variational Information Maximizing Exploration

Classifying Options for Deep Reinforcement Learning

A Deep Hierarchical Approach to Lifelong Learning in Minecraft

Benchmarking Deep Reinforcement Learning for Continuous Control

Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation

Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection

Deep Reinforcement Learning from Self-Play in Imperfect-Information Games

Continuous Deep Q-Learning with Model-based Acceleration

Deep Exploration via Bootstrapped DQN

Value Iteration Networks

Learning to Communicate to Solve Riddles with Deep Distributed Recurrent Q-Networks

Asynchronous Methods for Deep Reinforcement Learning

Mastering the game of Go with deep neural networks and tree search