Hardware Conditioned Policies for Multi-Robot Transfer Learning (2018-11-24T00:00:00.000000Z)

TL;DR

This work uses the kinematic structure directly as the hardware encoding and shows great zero-shot transfer to completely novel robots not seen during training and demonstrates that fine-tuning the policy network is significantly more sample-efficient than training a model from scratch.

Abstract

Deep reinforcement learning could be used to learn dexterous robotic policies but it is challenging to transfer them to new robots with vastly different hardware properties. It is also prohibitively expensive to learn a new policy from scratch for each robot hardware due to the high sample complexity of modern state-of-the-art algorithms. We propose a novel approach called Hardware Conditioned Policies where we train a universal policy conditioned on a vector representation of robot hardware. We considered robots in simulation with varied dynamics, kinematic structure, kinematic lengths and degrees-of-freedom. First, we use the kinematic structure directly as the hardware encoding and show great zero-shot transfer to completely novel robots not seen during training. For robots with lower zero-shot success rate, we also demonstrate that fine-tuning the policy network is significantly more sample-efficient than training a model from scratch. In tasks where knowing the agent dynamics is important for success, we learn an embedding for robot hardware and show that policies conditioned on the encoding of hardware tend to generalize and transfer well. Videos of experiments are available at: https://sites.google.com/view/robot-transfer-hcp.

Authors

A. Gupta

21 papers

Tao Chen

1 papers

Adithyavairavan Murali

2 papers

TL;DR

Abstract

Authors

References37 items

Graph networks as learnable physics engines for inference and control

NerveNet: Learning Structured Policy with Graph Neural Networks

Sim-to-Real Transfer of Robotic Control with Dynamics Randomization

Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments

Adversarially Robust Policy Learning: Active construction of physically-plausible perturbations

CASSL: Curriculum Accelerated Self-Supervised Learning

Multi-robot transfer learning: A dynamical system perspective

Proximal Policy Optimization Algorithms

Hindsight Experience Replay

Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics

Domain randomization for transferring deep neural networks from simulation to the real world

Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning

Preparing for the Unknown: Learning a Universal Policy with Online System Identification

EPOpt: Learning Robust Neural Network Policies Using Model Ensembles

Learning to push by grasping: Using multiple tasks for effective learning

Learning modular neural network policies for multi-task and multi-robot transfer

Progressive Neural Networks

PLATO: Policy learning using adaptive trajectory optimization

Mastering the game of Go with deep neural networks and tree search

Ensemble-CIO: Full-body dynamic motion planning that transfers to physical humanoids

One-shot learning of manipulation skills with online dynamics adaptation and neural network priors

Continuous control with deep reinforcement learning

Universal Value Function Approximators

End-to-End Training of Deep Visuomotor Policies

Human-level control through deep reinforcement learning

Alignment-based transfer learning for robot models

MuJoCo: A physics engine for model-based control

Design and development of a 7-DOF humanoid arm

Infinite-Horizon Model Predictive Control for Periodic Tasks with Contacts

Transfer learning for reinforcement learning on a physical robot

Transfer Learning for Reinforcement Learning Domains: A Survey

State transition, balancing, station keeping, and yaw control for a dynamically stable single spherical wheel mobile robot

Using inaccurate models in reinforcement learning

Robust Control of Markov Decision Processes with Uncertain Transition Matrices

Adaptive manipulator control a case study

Discrete and Continuous Bang-Bang and Facial Spaces Or: Look for the Extreme Points

System identiﬁcation and modeling for mabel, a bipedal robot with a cable-differential-based compliant drivetrain

Field of Study

Venue Information

Name

Type

URL

Alternate Names