Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapes (2021-10-12T00:00:00.000000Z)

TL;DR

This work investigates what choices matter for learning such general vision-based agents in simulation, and what affects optimal transfer to the real robot, and then leverages data collected by such policies and improves upon them with offline RL.

Abstract

We study the problem of robotic stacking with objects of complex geometry. We propose a challenging and diverse set of such objects that was carefully designed to require strategies beyond a simple"pick-and-place"solution. Our method is a reinforcement learning (RL) approach combined with vision-based interactive policy distillation and simulation-to-reality transfer. Our learned policies can efficiently handle multiple object combinations in the real world and exhibit a large variety of stacking skills. In a large experimental study, we investigate what choices matter for learning such general vision-based agents in simulation, and what affects optimal transfer to the real robot. We then leverage data collected by such policies and improve upon them with offline RL. A video and a blog post of our work are provided as supplementary material.

Authors

Martin A. Riedmiller

7 papers

R. Hadsell

10 papers

A. Abdolmaleki

5 papers

TL;DR

Abstract

Authors

References71 items

Active Learning of Abstract Plan Feasibility

From Rocks to Walls: a Model-free Reinforcement Learning Approach to Dry Stacking with Irregular Rocks

RetinaGAN: An Object-aware Approach to Sim-to-Real Transfer

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Learning Dexterous Manipulation from Suboptimal Experts

Critic Regularized Regression

Accelerating Online Reinforcement Learning with Offline Datasets

RL-CycleGAN: Reinforcement Learning Aware Simulation-to-Real

Towards Practical Multi-Object Manipulation using Relational Reinforcement Learning

Self-Supervised Sim-to-Real Adaptation for Visual Robotic Manipulation

Adaptive Curriculum Generation from Demonstrations for Sim-to-Real Visuomotor Control

A Framework for Data-Driven Robotics

“Good Robot!”: Efficient Reinforcement Learning for Multi-Step Visual Tasks with Sim to Real Transfer

ROBEL: Robotics Benchmarks for Learning with Low-Cost Robots

Deep Dynamics Models for Learning Dexterous Manipulation

Adversarial Grasp Objects

Transformer-XL: Attentive Language Models beyond a Fixed-Length Context

Sim-To-Real via Sim-To-Sim: Data-Efficient Robotic Grasping via Randomized-To-Canonical Adaptation Networks

Off-Policy Deep Reinforcement Learning without Exploration

Relative Entropy Regularized Policy Iteration

Deep Q-Learning for Dry Stacking Irregular Objects

Learning dexterous in-hand manipulation

QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation

Relational inductive bias for physical construction in humans and machines

ShapeStacks: Learning Vision-Based Physical Intuition for Generalised Object Stacking

Learning by Playing - Solving Sparse Reward Tasks from Scratch

Reinforcement and Imitation Learning for Diverse Visuomotor Skills

Maximum a Posteriori Policy Optimisation

OSQP: an operator splitting solver for quadratic programs

Sim-to-Real Transfer of Robotic Control with Dynamics Randomization

Asymmetric Actor Critic for Image-Based Robot Learning

Domain Randomization and Generative Models for Robotic Grasping

Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching

A Distributional Perspective on Reinforcement Learning

Transferring End-to-End Visuomotor Control from Simulation to Real World for a Multi-Stage Task

Attention is All you Need

Autonomous robotic stone stacking with online next best object target pose planning

Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics

One-Shot Imitation Learning

Domain randomization for transferring deep neural networks from simulation to the real world

Adversarial Discriminative Domain Adaptation

(CAD)$^2$RL: Real Single-Image Flight without a Single Real Image

Safe and Efficient Off-Policy Reinforcement Learning

Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection

Learning Physical Intuition of Block Towers by Example

Analysis and Observations From the First Amazon Picking Challenge

Efficient self-collision avoidance based on focus of interest for humanoid robots

Policy Distillation

Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours

Domain-Adversarial Training of Neural Networks

OpenSoT: A whole-body control library for the compliant humanoid robot COMAN

Human-level control through deep reinforcement learning

Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocols

Adam: A Method for Stochastic Optimization

Vision guided robotic block stacking

MuJoCo: A physics engine for model-based control

ImageNet classification with deep convolutional neural networks

A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning

A versatile Generalized Inverted Kinematics implementation for collaborative working humanoid robots: The Stack Of Tasks

A Local Collision Avoidance Method for Non-strictly Convex Polyhedra

A Mathematical Introduction to Robotic Manipulation

Planning optimal grasps

The mechanics of manipulation

Visual pattern recognition by moment invariants

Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping

Triangulation

A Framework for Behavioural Cloning

Ieee Transactions on Pattern Analysis and Machine Intelligence 1 3d Reasoning from Blocks to Stability

(a) Real-world image sequences without image augmentations. (b) Real-world image sequences with image augmentations

End: Opens and lifts the gripper to an absolute height of 30 cm . Final state.

Move closed gripper to bottom object stack position: Moves the TCP to a point 3 cm above the bottom object while maintaining the gripper closed

Field of Study

Venue Information

Name

Type

URL