Adversarial Skill Networks: Unsupervised Robot Skill Learning from Video (2019-10-21T00:00:00.000000Z)

TL;DR

This work proposes a novel approach to learn a task-agnostic skill embedding space from unlabeled multi-view videos by using an adversarial loss, and shows that the learned embedding enables training of continuous control policies to solve novel tasks that require the interpolation of previously seen skills.

Abstract

Key challenges for the deployment of reinforcement learning (RL) agents in the real world are the discovery, representation and reuse of skills in the absence of a reward function. To this end, we propose a novel approach to learn a task-agnostic skill embedding space from unlabeled multi-view videos. Our method learns a general skill embedding independently from the task context by using an adversarial loss. We combine a metric learning loss, which utilizes temporal video coherence to learn a state representation, with an entropy-regularized adversarial skill-transfer loss. The metric learning loss learns a disentangled representation by attracting simultaneous viewpoints of the same observations and repelling visually similar frames from temporal neighbors. The adversarial skill-transfer loss enhances re-usability of learned skill embeddings over multiple task domains. We show that the learned embedding enables training of continuous control policies to solve novel tasks that require the interpolation of previously seen skills. Our extensive evaluation with both simulation and real world data demonstrates the effectiveness of our method in learning transferable skills from unlabeled interaction videos and composing them for new tasks. Code, pretrained models and dataset are available at http://robotskills.cs.uni-freiburg.de

Authors

Wolfram Burgard

22 papers

Oier Mees

9 papers

Markus Merklinger

1 papers

TL;DR

Abstract

Authors

References33 items

Generative Adversarial Networks

Adaptive Curriculum Generation from Demonstrations for Sim-to-Real Visuomotor Control

Self-supervised 3D Shape and Viewpoint Estimation from Single Images for Robotics

Hierarchical Reinforcement Learning for Concurrent Discovery of Compound and Composable Policies

End-to-End Robotic Reinforcement Learning without Reward Engineering

Unsupervised Visuomotor Control through Distributional Planning Networks

MONet: Unsupervised Scene Decomposition and Representation

A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play

Reward learning from human preferences and demonstrations in Atari

Learning Actionable Representations from Visual Observations

Learning dexterous in-hand manipulation

Playing hard exploration games by watching YouTube

DeepMimic

Composable Deep Reinforcement Learning for Robotic Manipulation

Distributed Distributional Deterministic Policy Gradients

Learning an Embedding Space for Transferable Robot Skills

Rainbow: Combining Improvements in Deep Reinforcement Learning

Proximal Policy Optimization Algorithms

PVEs: Position-Velocity Encoders for Unsupervised Learning of Structured State Representations

Conditional Adversarial Domain Adaptation

Time-Contrastive Networks: Self-Supervised Learning from Video

Adversarial Discriminative Domain Adaptation

Unsupervised Perceptual Rewards for Imitation Learning

Sim-to-Real Robot Learning from Pixels with Progressive Nets

Visual closed-loop control for pouring liquids

Deep visual foresight for planning robot motion

Rethinking the Inception Architecture for Computer Vision

Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks

Deep Metric Learning via Lifted Structured Feature Embedding

Deep spatial autoencoders for visuomotor learning

Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images

Unsupervised Domain Adaptation by Backpropagation

Auto-Encoding Variational Bayes

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names