Adaptive Risk-Tendency: Nano Drone Navigation in Cluttered Environments with Distributional Reinforcement Learning (2022-03-28T00:00:00.000000Z)

TL;DR

This paper investigates a specific case where a nano quadcopter robot learns to navigate an apriori-unknown cluttered environment under partial observability and presents a distributional reinforcement learning framework to generate adaptive risk-tendency policies.

Abstract

Enabling the capability of assessing risk and making risk-aware decisions is essential to applying reinforcement learning to safety-critical robots like drones. In this paper, we investigate a specific case where a nano quadcopter robot learns to navigate an apriori-unknown cluttered environment under partial observability. We present a distributional reinforcement learning framework to generate adaptive risk-tendency policies. Specifically, we propose to use lower tail conditional variance of the learnt return distribution as intrinsic uncertainty estimation, and use exponentially weighted average forecasting (EWAF) to adapt the risk-tendency in accordance with the estimated uncertainty. In simulation and real-world empirical results, we show that (1) the most effective risk-tendency varies across states, (2) the agent with adaptive risk-tendency achieves superior performance compared to risk-neutral policy or risk-averse policy baselines. Code and video can be found in this repository: https://github.com/tudelft/risk-sensitive-rl.git

Authors

Cheng Liu

1 papers

E. Kampen

1 papers

G. D. Croon

1 papers

TL;DR

Abstract

Authors

References43 items

Minimizing Safety Interference for Safe and Comfortable Automated Driving with Distributional Reinforcement Learning

Risk-Conditioned Distributional Soft Actor-Critic for Risk-Sensitive Navigation

Risk-Averse Offline Reinforcement Learning

Distributional Soft Actor Critic for Risk Sensitive Learning

A distributional code for value in dopamine-based reinforcement learning

Worst Cases Policy Gradients

Learning Resilient Behaviors for Navigation Under Uncertainty

Conditional value‐at‐risk beyond finance: a survey

Bio-inspired synthesis of xishacorenes A, B, and C, and a new congener from fuscol

Estimating Risk and Uncertainty in Deep Reinforcement Learning

Distributional Reinforcement Learning for Efficient Exploration

An Introduction to Deep Reinforcement Learning

Safe Reinforcement Learning With Model Uncertainty Estimates

Implicit Quantile Networks for Distributional Reinforcement Learning

Deep Learning using Rectified Linear Units (ReLU)

Distributional Reinforcement Learning with Quantile Regression

How Should a Robot Assess Risk? Towards an Axiomatic Theory of Risk in Robotics

A Distributional Perspective on Reinforcement Learning

Domain randomization for transferring deep neural networks from simulation to the real world

Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation

Uncertainty-Aware Reinforcement Learning for Collision Avoidance

Deep Exploration via Bootstrapped DQN

Continuous control with deep reinforcement learning

Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning

End-to-End Training of Deep Visuomotor Policies

Human-level control through deep reinforcement learning

Adam: A Method for Stochastic Optimization

On the Properties of Neural Machine Translation: Encoder–Decoder Approaches

Model Predictive Control

Prediction, learning, and games

Robust Statistics

Long Short-Term Memory

Q-learning

Book Reviews

Deep Reinforcement Learning with Dynamic Optimism

and s

Partially Observable Markov Decision Processes

I and J

A. and Q

Influence of cultivation temperature on the ligninolytic activity of selected fungal strains

Recent Advances in Quantile Regression Models: A Practical Guideline for Empirical Research

Crazyflie 2.1 Specifications

OpenAI

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names