Rainbow: Combining Improvements in Deep Reinforcement Learning (2017-10-06T00:00:00.000000Z)

TL;DR

This paper examines six extensions to the DQN algorithm and empirically studies their combination, showing that the combination provides state-of-the-art performance on the Atari 2600 benchmark, both in terms of data efficiency and final performance.

Abstract

The deep reinforcement learning community has made several independent improvements to the DQN algorithm. However, it is unclear which of these extensions are complementary and can be fruitfully combined. This paper examines six extensions to the DQN algorithm and empirically studies their combination. Our experiments show that the combination provides state-of-the-art performance on the Atari 2600 benchmark, both in terms of data efficiency and final performance. We also provide results from a detailed ablation study that shows the contribution of each component to overall performance.

Authors

David Silver

21 papers

H. V. Hasselt

5 papers

Georg Ostrovski

4 papers

TL;DR

Abstract

Authors

References33 items

A Distributional Perspective on Reinforcement Learning

Noisy Networks for Exploration

Evolution Strategies as a Scalable Alternative to Reinforcement Learning

FeUdal Networks for Hierarchical Reinforcement Learning

Learning to Repeat: Fine Grained Action Repetition for Deep Reinforcement Learning

Reinforcement Learning with Unsupervised Auxiliary Tasks

Learning to Act by Predicting the Future

Learning to Play in a Day: Faster Deep Reinforcement Learning by Optimality Tightening

PGQ: Combining policy gradient and Q-learning

Model-Free Episodic Control

Deep Successor Reinforcement Learning

Unifying Count-Based Exploration and Intrinsic Motivation

Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation

Learning values across many orders of magnitude

Deep Exploration via Bootstrapped DQN

Asynchronous Methods for Deep Reinforcement Learning

Dueling Network Architectures for Deep Reinforcement Learning

Prioritized Experience Replay

Deep Reinforcement Learning with Double Q-Learning

Deep Recurrent Q-Learning for Partially Observable MDPs

Massively Parallel Methods for Deep Reinforcement Learning

Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models

The Arcade Learning Environment: An Evaluation Platform for General Agents (Extended Abstract)

Human-level control through deep reinforcement learning

Trust Region Policy Optimization

Adam: A Method for Stochastic Optimization

Playing Atari with Deep Reinforcement Learning

The Arcade Learning Environment: An Evaluation Platform for General Agents

Double Q-learning

Self-improving reactive agents based on reinforcement learning, planning and teaching

Learning to predict by the methods of temporal differences

Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude

Reinforcement Learning: An Introduction

Field of Study

Venue Information

Name

Type

URL

Alternate Names