Benchmarking Deep Reinforcement Learning for Continuous Control

Published in

International Conference on Machine Learning(2016)

External Links:

Generate Graph

TL;DR

This work presents a benchmark suite of continuous control tasks, including classic tasks like cart-pole swing-up, tasks with very high state and action dimensionality such as 3D humanoid locomotion, task with partial observations, and tasks with hierarchical structure.

Abstract

Recently, researchers have made significant progress combining the advances in deep learning for learning feature representations with reinforcement learning. Some notable examples include training agents to play Atari games based on raw pixel data and to acquire advanced manipulation skills using raw sensory inputs. However, it has been difficult to quantify progress in the domain of continuous control due to the lack of a commonly adopted benchmark. In this work, we present a benchmark suite of continuous control tasks, including classic tasks like cart-pole swing-up, tasks with very high state and action dimensionality such as 3D humanoid locomotion, tasks with partial observations, and tasks with hierarchical structure. We report novel findings based on the systematic evaluation of a range of implemented reinforcement learning algorithms. Both the benchmark and reference implementations are released at this https URL in order to facilitate experimental reproducibility and to encourage adoption by other researchers.

Authors

P. Abbeel

35 papers

Yan Duan

5 papers

Xi Chen

4 papers

References86 items

Dynamic Programming

Memory-based control with recurrent neural networks

RLLib: C++ Library to Predict, Control, and Represent Learnable Knowledge Using On/Off Policy Reinforcement Learning

Learning Continuous Control Policies by Stochastic Value Gradients

Continuous control with deep reinforcement learning

Benchmarking Deep Reinforcement Learning for Continuous Control

Published in

International Conference on Machine Learning(2016)

External Links:

Generate Graph

TL;DR

Abstract

Authors

P. Abbeel

35 papers

Yan Duan

5 papers

Xi Chen

4 papers

References86 items

Dynamic Programming

Memory-based control with recurrent neural networks

RLLib: C++ Library to Predict, Control, and Represent Learnable Knowledge Using On/Off Policy Reinforcement Learning

Learning Continuous Control Policies by Stochastic Value Gradients

Continuous control with deep reinforcement learning

Rein Houthooft

2 papers

John Schulman

14 papers

Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images

High-Dimensional Continuous Control Using Generalized Advantage Estimation

Learning of Non-Parametric Control Policies with High-Dimensional State Features

Policy evaluation with temporal differences: a survey and comparison

End-to-End Training of Deep Visuomotor Policies

Human-level control through deep reinforcement learning

Trust Region Policy Optimization

Deep Learning for Real-Time Atari Game Play Using Offline Monte-Carlo Tree Search Planning

The Reinforcement Learning Competition 2014

dotRL: A platform for rapid Reinforcement Learning methods development and validation

A Survey on Policy Search for Robotics

The Open-Source TEXPLORE Code Release for Reinforcement Learning on Robots

Guided Policy Search

MuJoCo: A physics engine for model-based control

Synthesis and stabilization of complex behaviors through online trajectory optimization

ImageNet classification with deep convolutional neural networks

Deep Neural Networks for Acoustic Modeling in Speech Recognition

The Arcade Learning Environment: An Evaluation Platform for General Agents

Infinite-Horizon Model Predictive Control for Periodic Tasks with Contacts

SkyAI: Highly modularized reinforcement learning library

Relative Entropy Policy Search

ImageNet: A large-scale hierarchical image database

Life at Low Reynolds Number

Reinforcement learning of motor skills with policy gradients

Learning to Control a 6-Degree-of-Freedom Walking Robot

Solving Deep Memory POMDPs with Recurrent Policy Gradients

Automated directory assistance system - from theory to practice

Reinforcement learning by reward-weighted regression for operational space control

Learning Tetris Using the Noisy Cross-Entropy Method

One-shot learning of object categories

Covariant policy search

ε-mdps: learning in varying environments

Reinforcement Learning Using Neural Networks, with Applications to Motor Control

A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics

Completely Derandomized Self-Adaptation in Evolution Strategies

Reinforcement Learning with Long Short-Term Memory

A Natural Policy Gradient

The aurora experimental framework for the performance evaluation of speech recognition systems under noisy conditions

Stochastic real-valued reinforcement learning to solve a nonlinear control problem

The Cross-Entropy Method for Combinatorial and Continuous Optimization

Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning

Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition

2-D Pole Balancing with Recurrent Evolutionary Networks

Reinforcement Learning with Hierarchies of Machines

Long Short-Term Memory

Temporal Difference Learning and TD-Gammon

Swinging up the Acrobot: an example of intelligent control

DARPA TIMIT:: acoustic-phonetic continuous speech corpus CD-ROM, NIST speech disc 1-1.1

Darpa Timit Acoustic-Phonetic Continuous Speech Corpus CD-ROM {TIMIT} | NIST

SWITCHBOARD: telephone speech corpus for research and development

Animation of dynamic legged locomotion

Computer control of a double inverted pendulum

Pattern Recognition and Adaptive Control

The reinforcement learning competition

BOXES: AN EXPERIMENT IN ADAPTIVE CONTROL

TIMIT Acoustic-Phonetic Continuous Speech Corpus

CLS2: Closed loop simulation system

Box2D: A 2D physics engine for games

Maja machine learning framework. http://mloss.org/software/view

A Case Study in Approximate Linearization: The Acrobot Example

SkyAI: Highly modu- Benchmarking Deep Reinforcement Learning for Continuous Control larized reinforcement learning library

J. Mach. Learn. Res

ApproxRL: A Matlab toolbox for approximate RL and DP. http://busoniu.net/files/repository/ readme-approxrl.html

Learning Multiple Layers of Features from Tiny Images

Beliefbox: A framework for statistical methods in sequential decision making

Dynamical principles for neuroscience and intelligent biomimetic devices

SOLVING PARTIALLY OBSERVABLE REINFORCEMENT LEARNING PROBLEMS WITH RECURRENT NEURAL NETWORKS

Keepaway Soccer: From Machine Learning Testbed to Benchmark

Reinforcement Learning Benchmarks and Bake-offs II A workshop at the 2005 NIPS conference

The mnist database of handwritten digits

Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning

Policy Gradient Methods for Robot Control

Reinforcement Learning in Continuous Time and Space

Neuro-Dynamic Programming : An Overview

Efficient memory-based learning for robot control

3-D balance in legged locomotion: modeling and simulation for the one-legged case

3D balance in legged locomotion: modeling and simulation for the one-legged case (abstract only)

Error decorrelation: a technique for matching a class of functions

On induced stability

Noname manuscript No. (will be inserted by the editor) Policy Search for Motor Primitives in Robotics

Edinburgh Research Explorer The PASCAL Visual Object Classes (VOC) Challenge

Field of Study

Computer ScienceMathematics

Venue Information

Name

International Conference on Machine Learning

Type

conference

URL

https://icml.cc/

Alternate Names

ICML
Int Conf Mach Learn