HeR-DRL:Heterogeneous Relational Deep Reinforcement Learning for Single-Robot and Multi-Robot Crowd Navigation

Abstract

Crowd navigation has garnered significant research attention in recent years, particularly with the advent of DRL-based methods. Current DRL-based methods have extensively explored interaction relationships in single-robot scenarios. However, the heterogeneity of multiple interaction relationships is often disregarded. This “interaction blind spot” hinders progress towards more complex scenarios, such as multi-robot crowd navigation. In this letter, we propose a heterogeneous relational deep reinforcement learning method, named HeR-DRL, which utilizes a customized heterogeneous Graph Neural Network (GNN) to enhance overall performance in crowd navigation. Firstly, we devised a method for constructing robot-crowd heterogenous relation graph that effectively simulates the heterogeneous pair-wise interaction relationships. Based on this graph, we proposed a novel heterogeneous GNN to encode interaction relationship information. Finally, we incorporate the encoded information into deep reinforcement learning to explore the optimal policy. HeR-DRL is rigorously evaluated by comparing it to state-of-the-art algorithms in both single-robot and multi-robot circle crossing scenarios. The experimental results demonstrate that HeR-DRL surpasses the state-of-the-art approaches in overall performance, particularly excelling in terms of efficiency and comfort. This underscores the significance of heterogeneous interactions in crowd navigation.