Visual Robot Task Planning (2018-03-30T00:00:00.000000Z)

TL;DR

A neural network architecture and associated planning algorithm that learns a representation of the world that can generate prospective futures, uses this generative model to simulate the result of sequences of high-level actions in a variety of environments, and evaluates these actions via a variant of Monte Carlo Tree Search to find a viable solution to a particular problem.

Abstract

Prospection is key to solving challenging problems in new environments, but it has not been deeply explored as applied to task planning for perception-driven robotics. We propose visual robot task planning, where we take in an input image and must generate a sequence of high-level actions and associated observations that achieve some task. In this paper, we describe a neural network architecture and associated planning algorithm that (1) learns a representation of the world that can generate prospective futures, (2) uses this generative model to simulate the result of sequences of high-level actions in a variety of environments, and (3) evaluates these actions via a variant of Monte Carlo Tree Search to find a viable solution to a particular problem. Our approach allows us to visualize intermediate motion goals and learn to plan complex activity from visual information, and used this to generate and visualize task plans on held-out examples of a block-stacking simulation.

Authors

R. Arora

3 papers

Gregory Hager

11 papers

Chris Paxton

3 papers

TL;DR

Abstract

Authors

References32 items

Neural Task Programming: Learning to Generalize Across Hierarchical Tasks

SE3-Pose-Nets: Structured Deep Dynamics Models for Visuomotor Planning and Control

Photographic Image Synthesis with Cascaded Refinement Networks

DARLA: Improving Zero-Shot Transfer in Reinforcement Learning

Imagination-Augmented Agents for Deep Reinforcement Learning

Learning model-based planning from scratch

Combining neural networks and tree search for task and motion planning in challenging environments

Deep predictive policy training using reinforcement learning

QMDP-Net: Deep Learning for Planning under Partial Observability

Wasserstein GAN

Improved Texture Networks: Maximizing Quality and Diversity in Feed-Forward Stylization and Texture Synthesis

Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypotheses

Image-to-Image Translation with Conditional Adversarial Networks

CoSTAR: Instructing collaborative robots with behavior trees and vision

Deep visual foresight for planning robot motion

Act to See and See to Act: POMDP planning for objects search in clutter

Strategic Attentive Writer for Learning Macro-Actions

Generative Adversarial Imitation Learning

Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning

Unsupervised Learning for Physical Interaction through Video Prediction

Analysis of the Structure of Surgical Activity for a Suturing and Knot-Tying Task

Value Iteration Networks

Discriminative Regularization for Generative Models

Robobarista: Learning to Manipulate Novel Objects via Deep Multimodal Embedding

Deep spatial autoencoders for visuomotor learning

Logic-Geometric Programming: An Optimization-Based Approach to Combined Task and Motion Planning

DeepMPC: Learning Deep Latent Features for Model Predictive Control

End-to-End Training of Deep Visuomotor Policies

Navigating Into the Future or Driven by the Past

Cognition-Enabled Autonomous Robot Control for the Realization of Home Chore Task Intelligence

Keras

PDDL-the planning domain definition language

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names