The CoSTAR Block Stacking Dataset: Learning with Workspace Constraints

Published in

IEEE/RJS International Conference on Intelligen...(2018)

External Links:

Generate Graph DownloadPDF

TL;DR

It is shown that a mild relaxation of the task and workspace constraints implicit in existing object grasping datasets can cause neural network based grasping algorithms to fail on even a simple block stacking task when executed under more realistic circumstances.

Abstract

A robot can now grasp an object more effectively than ever before, but once it has the object what happens next? We show that a mild relaxation of the task and workspace constraints implicit in existing object grasping datasets can cause neural network based grasping algorithms to fail on even a simple block stacking task when executed under more realistic circumstances. To address this, we introduce the JHU CoSTAR Block Stacking Dataset (BSD), where a robot interacts with 5.1 cm colored blocks to complete an order-fulfillment style block stacking task. It contains dynamic scenes and real time-series data in a less constrained environment than comparable datasets. There are nearly 12,000 stacking attempts and over 2 million frames of real data. We discuss the ways in which this dataset provides a valuable resource for a broad range of other topics of investigation. We find that hand-designed neural networks that work on prior datasets do not generalize to this task. Thus, to establish a baseline for this dataset, we demonstrate an automated search of neural network based models using a novel multiple-input HyperTree MetaModel, and find a final model which makes reasonable 3D pose predictions for grasping and stacking on our dataset. The CoSTAR BSD, code, and instructions are available at sites.google.com/site/costardataset.

Authors

Gregory Hager

11 papers

Andrew Hundt

1 papers

Varun Jain

1 papers

References44 items

Mechanical Search: Multi-Step Retrieval of a Target Object Occluded by Clutter

Sim-To-Real via Sim-To-Sim: Data-Efficient Robotic Grasping via Randomized-To-Canonical Adaptation Networks

Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitation

Robustness via Retrying: Closed-Loop Robotic Manipulation with Self-Supervised Learning

Evaluating Methods for End-User Creation of Robot Task Plans

The CoSTAR Block Stacking Dataset: Learning with Workspace Constraints

Published in

IEEE/RJS International Conference on Intelligen...(2018)

External Links:

Generate Graph DownloadPDF

TL;DR

Abstract

Authors

Gregory Hager

11 papers

Andrew Hundt

1 papers

Varun Jain

1 papers

References44 items

Mechanical Search: Multi-Step Retrieval of a Target Object Occluded by Clutter

Sim-To-Real via Sim-To-Sim: Data-Efficient Robotic Grasping via Randomized-To-Canonical Adaptation Networks

Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitation

Robustness via Retrying: Closed-Loop Robotic Manipulation with Self-Supervised Learning

Evaluating Methods for End-User Creation of Robot Task Plans

Chia-Hung Lin

1 papers

Chris Paxton

3 papers

A Real-time Robotic Grasp Approach with Oriented Anchor Box

Monocular Object Orientation Estimation using Riemannian Regression and Classification Networks

An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution

QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation

Playing hard exploration games by watching YouTube

ShapeStacks: Learning Vision-Based Physical Intuition for Generalised Object Stacking

Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approach

Jacquard: A Large Scale Dataset for Robotic Grasp Detection

Efficient Neural Architecture Search via Parameter Sharing

Self-Supervised Visual Planning with Temporal Skip Connections

Neural Task Programming: Learning to Generalize Across Hierarchical Tasks

Improved Regularization of Convolutional Neural Networks with Cutout

Learning Transferable Architectures for Scalable Image Recognition

End-to-End Learning of Semantic Grasping

Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics

Domain randomization for transferring deep neural networks from simulation to the real world

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

Deep Supervision with Shape Concepts for Occlusion-Aware 3D Object Parsing

Robotic grasp detection using deep convolutional neural networks

CoSTAR: Instructing collaborative robots with behavior trees and vision

Modular Deep Q Networks for Sim-to-real Transfer of Visuo-motor Policies

Learning modular neural network policies for multi-task and multi-robot transfer

Densely Connected Convolutional Networks

Learning to Poke by Poking: Experiential Learning of Intuitive Physics

Combining model-based policy search with online model learning for control of physical humanoids

Identity Mappings in Deep Residual Networks

Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection

Learning Physical Intuition of Block Towers by Example

Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning

Deep Residual Learning for Image Recognition

Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours

No More Pesky Learning Rate Guessing Games

Training Deeper Convolutional Networks with Deep Supervision

End-to-End Training of Deep Visuomotor Policies

Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

Real-time grasp detection using convolutional neural networks

Very Deep Convolutional Networks for Large-Scale Image Recognition

Deep learning for detecting robotic grasps

Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping

Field of Study

Computer Science

Journal Information

Name

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Page

1797-1804

Venue Information

Name

IEEE/RJS International Conference on Intelligent RObots and Systems

Type

conference

URL

http://www.iros.org/

Alternate Names

IROS
Intelligent Robots and Systems
Intell Robot Syst
IEEE/RJS Int Conf Intell Robot Syst