Learning Goal Embeddings via Self-Play for Hierarchical Reinforcement Learning (2018-11-22T00:00:00.000000Z)

TL;DR

This work proposes an unsupervised learning scheme, based on asymmetric self-play from Sukhbaatar et al. (2018), that automatically learns a good representation of sub-goals in the environment and a low-level policy that can execute them.

Abstract

In hierarchical reinforcement learning a major challenge is determining appropriate low-level policies. We propose an unsupervised learning scheme, based on asymmetric self-play from Sukhbaatar et al. (2018), that automatically learns a good representation of sub-goals in the environment and a low-level policy that can execute them. A high-level policy can then direct the lower one by generating a sequence of continuous sub-goal vectors. We evaluate our model using Mazebase and Mujoco environments, including the challenging AntGather task. Visualizations of the sub-goal embeddings reveal a logical decomposition of tasks within the environment. Quantitatively, our approach obtains compelling performance gains over non-hierarchical approaches.

Authors

Emily L. Denton

5 papers

Arthur Szlam

12 papers

R. Fergus

8 papers

Learning Goal Embeddings via Self-Play for Hierarchical Reinforcement Learning

TL;DR

Abstract

Authors

References21 items

Latent Space Policies for Hierarchical Reinforcement Learning

Diversity is All You Need: Learning Skills without a Reward Function

Learning an Embedding Space for Transferable Robot Skills

Unsupervised Learning of Goal Spaces for Intrinsically Motivated Goal Exploration

Proximal Policy Optimization Algorithms

Reverse Curriculum Generation for Reinforcement Learning

Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play

FeUdal Networks for Hierarchical Reinforcement Learning

Variational Intrinsic Control

The Option-Critic Architecture

Benchmarking Deep Reinforcement Learning for Continuous Control

MazeBase: A Sandbox for Learning from Games

Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning

MuJoCo: A physics engine for model-based control

What is Intrinsic Motivation? A Typology of Computational Approaches

Empowerment: a universal agent-centric measure of control

Intrinsically Motivated Reinforcement Learning

Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning

Curious model-building control systems

Lecture 6.5 -rmsprop, coursera: Neural networks for machine learning

Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names