Auxiliary Tasks Speed Up Learning PointGoal Navigation (2020-07-09T00:00:00.000000Z)

TL;DR

This work develops a method to significantly increase sample and time efficiency in learning PointNav using self-supervised auxiliary tasks (e.g. predicting the action taken between two egocentric observations, predicting the distance between two observations from a trajectory, etc.).

Abstract

PointGoal Navigation is an embodied task that requires agents to navigate to a specified point in an unseen environment. Wijmans et al. showed that this task is solvable but their method is computationally prohibitive, requiring 2.5 billion frames and 180 GPU-days. In this work, we develop a method to significantly increase sample and time efficiency in learning PointNav using self-supervised auxiliary tasks (e.g. predicting the action taken between two egocentric observations, predicting the distance between two observations from a trajectory,etc.).We find that naively combining multiple auxiliary tasks improves sample efficiency,but only provides marginal gains beyond a point. To overcome this, we use attention to combine representations learnt from individual auxiliary tasks. Our best agent is 5x faster to reach the performance of the previous state-of-the-art, DD-PPO, at 40M frames, and improves on DD-PPO's performance at40M frames by 0.16 SPL. Our code is publicly available at this https URL.

Authors

Dhruv Batra

43 papers

Abhishek Das

11 papers

Erik Wijmans

6 papers

TL;DR

Abstract

Authors

References54 items

Learning To Explore Using Active Neural Mapping

CURL: Contrastive Unsupervised Representations for Reinforcement Learning

Improved Baselines with Momentum Contrastive Learning

A Simple Framework for Contrastive Learning of Visual Representations

Are We Making Real Progress in Simulated Environments? Measuring the Sim2Real Gap in Embodied Visual Navigation

Self-Supervised Learning of Pretext-Invariant Representations

Momentum Contrast for Unsupervised Visual Representation Learning

DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames

Situational Fusion of Visual Representation for Visual Navigation

Shaping Belief States with Generative Environment Models for RL

Unsupervised State Representation Learning in Atari

Contrastive Multiview Coding

Learning Representations by Maximizing Mutual Information Across Views

Data-Efficient Image Recognition with Contrastive Predictive Coding

SplitNet: Sim2Sim and Task2Task Transfer for Embodied Visual Navigation

Embodied Question Answering in Photorealistic Environments With Point Cloud Perception

Habitat: A Platform for Embodied AI Research

Self-Monitoring Navigation Agent via Auxiliary Progress Estimation

Mid-Level Visual Representations Improve Generalization and Sample Efficiency for Learning Visuomotor Policies

Neural Modular Control for Embodied Question Answering

Neural Predictive Belief Representations

Learning deep representations by mutual information estimation and maximization

On Evaluation of Embodied Navigation Agents

Representation Learning with Contrastive Predictive Coding

Gibson Env: Real-World Perception for Embodied Agents

Taskonomy: Disentangling Task Transfer Learning

Semi-parametric Topological Memory for Navigation

IQA: Visual Question Answering in Interactive Environments

Embodied Question Answering

Vision-and-Language Navigation: Interpreting Visually-Grounded Navigation Instructions in Real Environments

Matterport3D: Learning from RGB-D Data in Indoor Environments

Proximal Policy Optimization Algorithms

Attention is All you Need

Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics

Curiosity-Driven Exploration by Self-Supervised Prediction

DeepMind Lab

Learning to Navigate in Complex Environments

Reinforcement Learning with Unsupervised Auxiliary Tasks

Context Encoders: Feature Learning by Inpainting

Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles

Colorful Image Colorization

Deep Residual Learning for Image Recognition

High-Dimensional Continuous Control Using Generalized Advantage Estimation

Unsupervised Visual Representation Learning by Context Prediction

Human-level control through deep reinforcement learning

Adam: A Method for Stochastic Optimization

The 2014 Nobel Prize in Physiology or Medicine: A Spatial Model for Cognitive Neuroscience

Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

Visualizing and Understanding Convolutional Networks

Et al

Learning Multiple Tasks with Kernel Methods

Adaptive Auxiliary Task Weighting for Reinforcement Learning

Unsupervised Learning of Visual Representations using Videos

and as an in

Field of Study

Venue Information

Name

Type

URL

Alternate Names