Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulation (2021-05-31T00:00:00.000000Z)

TL;DR

The Attention-driven Robotic Manipulation (ARM) algorithm is presented, which is a general manipulation algorithm that can be applied to a range of sparse-rewarded tasks, given only a small number of demonstrations.

Abstract

Despite the success of reinforcement learning methods, they have yet to have their breakthrough moment when applied to a broad range of robotic manipulation tasks. This is partly due to the fact that reinforcement learning algorithms are notoriously difficult and time consuming to train, which is exacerbated when training from images rather than full-state inputs. As humans perform manipulation tasks, our eyes closely monitor every step of the process with our gaze focusing sequentially on the objects being manipulated. With this in mind, we present our Attention-driven Robotic Manipulation (ARM) algorithm, which is a general manipulation algorithm that can be applied to a range of sparse-rewarded tasks, given only a small number of demonstrations. ARM splits the complex task of manipulation into a 3 stage pipeline: (1) a Q-attention agent extracts interesting pixel locations from RGB and point cloud inputs, (2) a next-best pose agent that accepts crops from the Q-attention agent and outputs poses, and (3) a control agent that takes the goal pose and outputs joint actions. We show that current learning algorithms fail on a range of RLBench tasks, whilst ARM is successful.

Authors

A. Davison

7 papers

Stephen James

3 papers

TL;DR

Abstract

Authors

References54 items

Generalization Through Hand-Eye Coordination: An Action Space for Learning Spatially-Invariant Visuomotor Control

Transporter Networks: Rearranging the Visual World for Robotic Manipulation

Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels

Tactical Decision-Making in Autonomous Driving by Reinforcement Learning with Uncertainty Estimation

MoreFusion: Multi-object Reasoning for 6D Pose Estimation from Volumetric Fusion

RLBench: The Robot Learning Benchmark & Learning Environment

Improving Sample Efficiency in Model-Free Reinforcement Learning from Images

PyRep: Bringing V-REP to Deep Robot Learning

SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewards

Estimating Risk and Uncertainty in Deep Reinforcement Learning

Perceptual Attention-based Predictive Control

TossingBot: Learning to Throw Arbitrary Objects With Residual Physics

Learning agile and dynamic motor skills for legged robots

DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion

Sim-To-Real via Sim-To-Sim: Data-Efficient Robotic Grasping via Randomized-To-Canonical Adaptation Networks

Soft Actor-Critic Algorithms and Applications

Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks

Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning

Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning

SWIRL: A sequential windowed inverse reinforcement learning algorithm for robot tasks with delayed rewards

QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation

Sim-to-Real Reinforcement Learning for Deformable Object Manipulation

Self-Attention Generative Adversarial Networks

Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approach

Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning

Addressing Function Approximation Error in Actor-Critic Methods

DeepMind Control Suite

Asymmetric Actor Critic for Image-Based Robot Learning

Overcoming Exploration in Reinforcement Learning with Demonstrations

Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations

Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards

Transferring End-to-End Visuomotor Control from Simulation to Real World for a Multi-Stage Task

Attention is All you Need

Deep predictive policy training using reinforcement learning

(CAD)$^2$RL: Real Single-Image Flight without a Single Real Image

Deep visual foresight for planning robot motion

3D Simulation for Robot Arm Control with Deep Q-Learning

Generative Adversarial Imitation Learning

OpenAI Gym

U-Net: Convolutional Networks for Biomedical Image Segmentation

End-to-End Training of Deep Visuomotor Policies

Human-level control through deep reinforcement learning

Show, Attend and Tell: Neural Image Caption Generation with Visual Attention

Neural Machine Translation by Jointly Learning to Align and Translate

The Open Motion Planning Library

Keyframe-based Learning from Demonstration

Et al

Machine Learning for Fast Quadrupedal Locomotion

The Roles of Vision and Eye Movements in the Control of Activities of Daily Living

Simple statistical gradient-following algorithms for connectionist reinforcement learning

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

V-REP: A versatile and scalable robot simulation framework

A Single-Query Bi-Directional Probabilistic Roadmap Planner with Lazy Collision Checking

Noname manuscript No. (will be inserted by the editor) Policy Search for Motor Primitives in Robotics

Field of Study

Journal Information

Name

Page

Volume

Venue Information

Name

Type

URL

Alternate Names