Trust Region Policy Optimization (2015-02-19T00:00:00.000000Z)

TL;DR

A method for optimizing control policies, with guaranteed monotonic improvement, by making several approximations to the theoretically-justified scheme, called Trust Region Policy Optimization (TRPO).

Abstract

In this article, we describe a method for optimizing control policies, with guaranteed monotonic improvement. By making several approximations to the theoretically-justified scheme, we develop a practical algorithm, called Trust Region Policy Optimization (TRPO). This algorithm is effective for optimizing large nonlinear policies such as neural networks. Our experiments demonstrate its robust performance on a wide variety of tasks: learning simulated robotic swimming, hopping, and walking gaits; and playing Atari games using images of the screen as input. Despite its approximations that deviate from the theory, TRPO tends to give monotonic improvement, with little tuning of hyperparameters.

Authors

P. Abbeel

35 papers

S. Levine

59 papers

Michael I. Jordan

14 papers

TL;DR

Abstract

Authors

References43 items

Proximal Policy Optimization Algorithms

Deep Learning for Real-Time Atari Game Play Using Offline Monte-Carlo Tree Search Planning

Learning Neural Network Policies with Guided Policy Search under Unknown Dynamics

Playing Atari with Deep Reinforcement Learning

Approximate Dynamic Programming Finally Performs Well in the Game of Tetris

A Survey on Policy Search for Robotics

Safe Policy Iteration

Revisiting Natural Gradient for Deep Networks

MuJoCo: A physics engine for model-based control

On the Complexity of Bandit and Derivative-Free Stochastic Convex Optimization

The Arcade Learning Environment: An Evaluation Platform for General Agents

Online Learning and Online Convex Optimization

Relative Entropy Policy Search

Optimal gait and form for animal locomotion

Reinforcement learning of motor skills with policy gradients

Reinforcement learning by reward-weighted regression for operational space control

Learning Tetris Using the Noisy Cross-Entropy Method

Fast biped walking with a reflexive controller and real-time policy searching

Simulation optimization: a review, new developments, and applications

Stochastic policy gradient reinforcement learning on a simple 3D biped

Numerical optimization

A Tutorial on MM Algorithms

Natural Actor-Critic

Reinforcement Learning as Classification: Leveraging Modern Classifiers

Covariant policy search

Approximately Optimal Approximate Reinforcement Learning

Infinite-Horizon Policy-Gradient Estimation

A Natural Policy Gradient

PEGASUS: A policy search method for large MDPs and POMDPs

Numerical Optimization (Springer Series in Operations Research and Financial Engineering)

Exponentiated Gradient Versus Gradient Descent for Linear Predictors

Dynamic Programming and Optimal Control. Volume 1

Adapting arbitrary normal mutation distributions in evolution strategies: the covariance matrix adaptation

Dynamic Programming and Optimal Control, Two Volume Set

Neuronlike adaptive elements that can solve difficult learning control problems

Markov chains and mixing times

Optimal rates for zero-order optimization: the power of two function evaluations

Monte Carlo theory, methods and examples

Training Deep and Recurrent Networks with Hessian-Free Optimization

EFFICIENT METHODS IN CONVEX PROGRAMMING

Asymptopia: an exposition of statistical asymptotic theory

Boekbespreking van D.P. Bertsekas (ed.), Dynamic programming and optimal control - volume 2

By averaging over samples

Field of Study

Venue Information

Name

Type

URL

Alternate Names