VUSFA: Variational Universal Successor Features Approximator to Improve Transfer DRL for Target Driven Visual Navigation

Published in

arXiv.org(2019)

External Links:

Generate Graph

TL;DR

This paper builds on the concept of Universal Successor Features with an A3C agent and introduces the novel architectural contribution of a Successor Feature Dependant Policy (SFDP) and adopts the idea of Variational Information Bottlenecks to achieve state of the art performance.

Abstract

In this paper, we show how novel transfer reinforcement learning techniques can be applied to the complex task of target driven navigation using the photorealistic AI2THOR simulator. Specifically, we build on the concept of Universal Successor Features with an A3C agent. We introduce the novel architectural contribution of a Successor Feature Dependant Policy (SFDP) and adopt the concept of Variational Information Bottlenecks to achieve state of the art performance. VUSFA, our final architecture, is a straightforward approach that can be implemented using our open source repository. Our approach is generalizable, showed greater stability in training, and outperformed recent approaches in terms of transfer learning ability.

Authors

Suranga Nanayakkara

2 papers

Shamane Siriwardhana

1 papers

Rivindu Weerasekera

1 papers

References34 items

Universal Successor Features Approximators

Universal Successor Features for Transfer Reinforcement Learning

Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow

CrowdMove: Autonomous Mapless Navigation in Crowded Scenarios

Transfer in Deep Reinforcement Learning Using Successor Features and Generalised Policy Improvement

VUSFA: Variational Universal Successor Features Approximator to Improve Transfer DRL for Target Driven Visual Navigation

Published in

arXiv.org(2019)

External Links:

Generate Graph

TL;DR

Abstract

Authors

Suranga Nanayakkara

2 papers

Shamane Siriwardhana

1 papers

Rivindu Weerasekera

1 papers

References34 items

Universal Successor Features Approximators

Universal Successor Features for Transfer Reinforcement Learning

Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow

CrowdMove: Autonomous Mapless Navigation in Crowded Scenarios

Transfer in Deep Reinforcement Learning Using Successor Features and Generalised Policy Improvement

Denys J. C. Matthies

1 papers

Learning to Navigate in Cities Without a Map

Universal Successor Representations for Transfer Reinforcement Learning

AI2-THOR: An Interactive 3D Environment for Visual AI

Mastering the game of Go without human knowledge

Deep reinforcement learning with successor features for navigation across similar environments

Target-driven visual navigation in indoor scenes using deep reinforcement learning

Successor Features for Transfer in Reinforcement Learning

Deep Successor Reinforcement Learning

Asynchronous Methods for Deep Reinforcement Learning

Mastering the game of Go with deep neural networks and tree search

Universal Value Function Approximators

Deep learning and the information bottleneck principle

Human-level control through deep reinforcement learning

Playing Atari with Deep Reinforcement Learning

Horde: a scalable real-time architecture for learning knowledge from unsupervised sensorimotor interaction

Transfer Learning for Reinforcement Learning Domains: A Survey

Towards human-centered support for indoor navigation

Visual Navigation for Mobile Robots: A Survey

Map-based navigation in mobile robots: I. A review of localization strategies

On-line map building and navigation for autonomous mobile robots

Improving Generalization for Temporal Difference Learning: The Successor Representation

Constrained Optimization and Lagrange Multiplier Methods

AUTO-ENCODING VARIATIONAL BAYES

Deep Variational Information Bottleneck

2017], who introduced Successor Features (SF)

Alemi et al., 2016] evaluated the method of supervised learning tasks

| k  into the loss function J ( q , E ) min with a Lagrangian Multiplier

Reinforcement Learning: An Introduction

The vector field histogram-fast obstacle avoidance for mobile robots

Field of Study

Computer ScienceMathematics

Journal Information

Name

ArXiv

Volume

abs/2005.00687

Venue Information

Name

arXiv.org

Type

URL

https://arxiv.org

Alternate Names

ArXiv