What Can I Do Here? Learning New Skills by Imagining Visual Affordances

Published in

IEEE International Conference on Robotics and A...(2021)

External Links:

Generate Graph DownloadPDF

TL;DR

This paper aims to study how generative models of possible outcomes can allow a robot to learn visual representations of affordances, so that the robot can sample potentially possible outcomes in new situations, and then further train its policy to achieve those outcomes.

Abstract

A generalist robot equipped with learned skills must be able to perform many tasks in many different environments. However, zero-shot generalization to new settings is not always possible. When the robot encounters a new environment or object, it may need to finetune some of its previously learned skills to accommodate this change. But crucially, previously learned behaviors and models should still be suitable to accelerate this relearning. In this paper, we aim to study how generative models of possible outcomes can allow a robot to learn visual representations of affordances, so that the robot can sample potentially possible outcomes in new situations, and then further train its policy to achieve those outcomes. In effect, prior data is used to learn what kinds of outcomes may be possible, such that when the robot encounters an unfamiliar setting, it can sample potential outcomes from its model, attempt to reach them, and thereby update both its skills and its outcome model. We show that this approach can be used to train goal-conditioned policies that operate on raw image inputs, and can rapidly learn to manipulate new objects via our proposed affordance-directed exploration scheme.

Authors

S. Levine

59 papers

Alexander Khazatsky

1 papers

Ashvin Nair

3 papers

References55 items

Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short Survey

Deep Affordance Foresight: Planning Through What Can Be Done in the Future

robosuite: A Modular Simulation Framework and Benchmark for Robot Learning

What can I do here? A Theory of Affordances in Reinforcement Learning

Accelerating Online Reinforcement Learning with Offline Datasets

What Can I Do Here? Learning New Skills by Imagining Visual Affordances

Published in

IEEE International Conference on Robotics and A...(2021)

External Links:

Generate Graph DownloadPDF

TL;DR

Abstract

Authors

S. Levine

59 papers

Alexander Khazatsky

1 papers

Ashvin Nair

3 papers

References55 items

Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short Survey

Deep Affordance Foresight: Planning Through What Can Be Done in the Future

robosuite: A Modular Simulation Framework and Benchmark for Robot Learning

What can I do here? A Theory of Affordances in Reinforcement Learning

Accelerating Online Reinforcement Learning with Offline Datasets

Contextual Imagined Goals for Self-Supervised Robotic Learning

Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables

Skew-Fit: State-Covering Self-Supervised Reinforcement Learning

Learning Latent Plans from Play

強化学習におけるVariational Information Maximizing Explorationに基づく状態探索の効率化

Unsupervised Control Through Non-Parametric Discriminative Rewards

Visual Reinforcement Learning with Imagined Goals

Data-Efficient Hierarchical Reinforcement Learning

Unsupervised Learning of Goal Spaces for Intrinsically Motivated Goal Exploration

Neural Discrete Representation Learning

A brief review of affordance in robotic manipulation research

Computational models of affordance in robotics: a taxonomy and systematic classification

Hindsight Experience Replay

Automatic Goal Generation for Reinforcement Learning Agents

Curiosity-Driven Exploration by Self-Supervised Prediction

Information theoretic MPC for model-based reinforcement learning

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning

RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning

Affordance Research in Developmental Robotics: A Survey

Learning to Poke by Poking: Experiential Learning of Intuitive Physics

Conditional Image Generation with PixelCNN Decoders

Unifying Count-Based Exploration and Intrinsic Motivation

Adversarial Feature Learning

Mastering the game of Go with deep neural networks and tree search

Learning Structured Output Representation using Deep Conditional Generative Models

Universal Value Function Approximators

Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models

End-to-End Training of Deep Visuomotor Policies

Auto-Encoding Variational Bayes

Playing Atari with Deep Reinforcement Learning

Active learning of inverse models with intrinsically motivated goal exploration in robots

ImageNet classification with deep convolutional neural networks

Exploration in Model-based Reinforcement Learning by Empirically Estimating Learning Progress

Learning Generalizable Control Programs

PILCO: A Model-Based and Data-Efficient Approach to Policy Search

Relative Entropy Policy Search

Intrinsically Motivated Reinforcement Learning

Machine Learning for Fast Quadrupedal Locomotion

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

GENERATIVE ADVERSARIAL NETS

“Visual Affordance and Function Understanding: A Survey,

The Developing Person Through The Life Span

Pybullet, a python module for physics simulation for games, robotics and machine learning

Toward Affordance-Aware Planning

Biped dynamic walking using reinforcement learning

Learning to Achieve Goals

The Ecological Approach to Visual Perception

The ecological approach

Noname manuscript No. (will be inserted by the editor) Policy Search for Motor Primitives in Robotics

Field of Study

Computer Science

Journal Information

Name

2021 IEEE International Conference on Robotics and Automation (ICRA)

Page

14291-14297

Venue Information

Name

IEEE International Conference on Robotics and Automation

Type

conference

URL

http://ieeexplore.ieee.org/xpl/conhome.jsp?punumber=1000639

Alternate Names

International Conference on Robotics and Automation
Int Conf Robot Autom
ICRA
IEEE Int Conf Robot Autom