Generative Adversarial Imitation Learning (2016-06-10T00:00:00.000000Z)

TL;DR

A new general framework for directly extracting a policy from data, as if it were obtained by reinforcement learning following inverse reinforcement learning, is proposed and a certain instantiation of this framework draws an analogy between imitation learning and generative adversarial networks.

Abstract

Consider learning a policy from example expert behavior, without interaction with the expert or access to reinforcement signal. One approach is to recover the expert's cost function with inverse reinforcement learning, then extract a policy from that cost function with reinforcement learning. This approach is indirect and can be slow. We propose a new general framework for directly extracting a policy from data, as if it were obtained by reinforcement learning following inverse reinforcement learning. We show that a certain instantiation of our framework draws an analogy between imitation learning and generative adversarial networks, from which we derive a model-free imitation learning algorithm that obtains significant performance gains over existing model-free methods in imitating complex behaviors in large, high-dimensional environments.

Authors

Stefano Ermon

21 papers

Jonathan Ho

4 papers

TL;DR

Abstract

Authors

References32 items

OpenAI Gym

Model-Free Imitation Learning with Policy Optimization

Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization

RLPy: a value-function-based reinforcement learning framework for education and research

High-Dimensional Continuous Control Using Generalized Advantage Estimation

Trust Region Policy Optimization

Adam: A Method for Stochastic Optimization

Learning Neural Network Policies with Guided Policy Search under Unknown Dynamics

Infinite Time Horizon Maximum Causal Entropy Inverse Reinforcement Learning

MuJoCo: A physics engine for model-based control

Continuous Inverse Optimal Control with Locally Optimal Examples

Nonlinear Inverse Reinforcement Learning with Gaussian Processes

A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning

Modeling Interaction via the Principle of Maximum Causal Entropy

Efficient Reductions for Imitation Learning

Learning to search: Functional gradient techniques for imitation learning

Maximum Entropy Inverse Reinforcement Learning

Apprenticeship learning using linear programming

A Game-Theoretic Approach to Apprenticeship Learning

Elements of Information Theory

Apprenticeship learning via inverse reinforcement learning

Algorithms for Inverse Reinforcement Learning

Markov Decision Processes: Discrete Stochastic Dynamic Programming

Convex analysis and minimization algorithms

Efficient Training of Artificial Neural Networks for Autonomous Navigation

Neuronlike adaptive elements that can solve difficult learning control problems

GENERATIVE ADVERSARIAL NETS

Infinite time horizon maximum causal entropy inverse reinforcement learning

Learning agents for uncertain environments

Efficient memory-based learning for robot control

The minimax principle in asymptotic statistical theory

© Institute of Mathematical Statistics, 2009 ON SURROGATE LOSS FUNCTIONS AND f-DIVERGENCES 1

Field of Study

Venue Information

Name

Type

URL

Alternate Names