Action Priors for Large Action Spaces in Robotics (2021-01-11T00:00:00.000000Z)

TL;DR

The results indicate that this approach can be used to solve robotic manipulation problems that would otherwise be infeasible without expert demonstrations.

Abstract

In robotics, it is often not possible to learn useful policies using pure model-free reinforcement learning without significant reward shaping or curriculum learning. As a consequence, many researchers rely on expert demonstrations to guide learning. However, acquiring expert demonstrations can be expensive. This paper proposes an alternative approach where the solutions of previously solved tasks are used to produce an action prior that can facilitate exploration in future tasks. The action prior is a probability distribution over actions that summarizes the set of policies found solving previous tasks. Our results indicate that this approach can be used to solve robotic manipulation problems that would otherwise be infeasible without expert demonstrations. Source code is available at https://github.com/ondrejba/action_priors.

Authors

Ondrej Biza

1 papers

Dian Wang

1 papers

Robert W. Platt

1 papers

TL;DR

Abstract

Authors

References39 items

Parrot: Data-Driven Behavioral Priors for Reinforcement Learning

Accelerating Reinforcement Learning with Learned Skill Priors

Policy learning in SE(3) action spaces

Form2Fit: Learning Shape Priors for Generalizable Assembly from Disassembly

Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning

InfoBot: Transfer and Exploration via the Information Bottleneck

Deictic Image Maps: An Abstraction For Learning Pose Invariant Manipulation Policies

Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning

Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching

Distral: Robust multitask reinforcement learning

Transferring End-to-End Visuomotor Control from Simulation to Real World for a Multi-Stage Task

On Calibration of Modern Neural Networks

Curiosity-Driven Exploration by Self-Supervised Prediction

Deep Q-learning From Demonstrations

Deep visual foresight for planning robot motion

Progressive Neural Networks

VIME: Variational Information Maximizing Exploration

Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection

Dueling Network Architectures for Deep Reinforcement Learning

Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning

Prioritized Experience Replay

Deep Reinforcement Learning with Double Q-Learning

Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images

U-Net: Convolutional Networks for Biomedical Image Segmentation

Goal-Based Action Priors

Action Priors for Learning Domain Invariances

Human-level control through deep reinforcement learning

What good are actions? Accelerating learning using learned action priors

Nonparametric Bayesian Policy Priors for Reinforcement Learning

Transfer Learning for Reinforcement Learning Domains: A Survey

An object-oriented representation for efficient reinforcement learning

Probabilistic policy reuse in a reinforcement learning agent

A theoretical analysis of Model-Based Interval Estimation

Improving Action Selection in MDP's via Knowledge Transfer

Catastrophic forgetting in connectionist networks

A Markovian Decision Process

Learning Action Priors for Visuomotor transfer

[Computer Graphics]: Three-Dimensional Graphics and Realism

Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence Bayesian Policy Search with Policy Priors

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names