Learning-Driven Exploration for Reinforcement Learning (2019-06-17T00:00:00.000000Z)

TL;DR

This work introduces entropy-based exploration (EBE) that enables an agent to explore efficiently the unexplored regions of state space and quantifies the agent's learning in a state using state-dependent action values and adaptively explores the state space.

Abstract

Effective and intelligent exploration remains an unresolved problem for reinforcement learning. Most contemporary reinforcement learning relies on simple heuristic strategies which are unable to intelligently distinguish the well-explored and the unexplored regions of state space, which can lead to inefficient use of training time. We introduce entropy-based exploration (EBE) that enables an agent to explore efficiently the unexplored regions of state space. EBE quantifies the agent's learning in a state using state-dependent action values and adaptively explores the state space, i.e. more exploration for the unexplored region of the state space. We perform experiments on a diverse set of environments and demonstrate that EBE enables efficient exploration that ultimatelyresults in faster learning withouthaving totune anyhyperparameter. The code to reproduce the experimentsisgiven at https://github.com/Usama1002/EBE-Exploration and the supplementary video is given at https://youtu.be/nJggIjjzKic.

Authors

Muhammad Usama

1 papers

D. Chang

1 papers

TL;DR

Abstract

Authors

References38 items

Blue River Controls: A toolkit for Reinforcement Learning Control Systems on Hardware

Deep Neural Networks in a Mathematical Framework

Diversity is All You Need: Learning Skills without a Reward Function

Diversity-Driven Exploration Strategy for Deep Reinforcement Learning

Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor

Proximal Policy Optimization Algorithms

Curiosity-Driven Exploration by Self-Supervised Prediction

Surprise-Based Intrinsic Motivation for Deep Reinforcement Learning

Reinforcement Learning with Deep Energy-Based Policies

PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications

#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning

Unifying Count-Based Exploration and Intrinsic Motivation

Curiosity-driven Exploration in Deep Reinforcement Learning via Bayesian Neural Networks

ViZDoom: A Doom-based AI research platform for visual reinforcement learning

VIME: Variational Information Maximizing Exploration

Deep Exploration via Bootstrapped DQN

Taming the Noise in Reinforcement Learning via Soft Updates

Dueling Network Architectures for Deep Reinforcement Learning

Prioritized Experience Replay

Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models

Human-level control through deep reinforcement learning

Trust Region Policy Optimization

Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

Learning and exploration in action-perception loops

An information-theoretic approach to curiosity-driven reinforcement learning

Planning to Be Surprised: Optimal Bayesian Exploration in Dynamic Environments

Relative Entropy Policy Search

Dynamic policy programming

Compositionality of optimal control laws

An analysis of model-based Interval Estimation for Markov Decision Processes

Linearly-solvable Markov decision problems

A possibility for implementing curiosity and boredom in model-building neural controllers

Asymptotically efficient adaptive allocation rules

Tensorboard smoothing implementation. https://github.com/tensorflow/tensorboard/blob/ f801ebf1f9fbfe2baee1ddd65714d0bccc640fb1/tensorboard/plugins/scalar/ vz_line_chart/vz-line-chart.ts#L704

Modeling Purposeful Adaptive Behavior with the Principle of Maximum Causal Entropy

Reinforcement Learning: An Introduction

REINFORCEMENT DRIVEN INFORMATION ACQUISITION IN NON-DETERMINISTIC ENVIRONMENTS

Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis Hassabis. Humanlevel control through deep reinforcement learning

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names