robots-2

PointGoal Navigation

3260 papers • 126 benchmarks • 313 datasets

This task has no description! Would you like to contribute one?

(Image credit: Papersgraph)

Benchmarks

These leaderboards are used to track progress in pointgoal-navigation-5

Trend

Dataset

Best Model

Actions

Gibson PointGoal Navigation

Libraries

i

Use these libraries to find pointgoal-navigation-5 models and implementations

facebookresearch/habitat-lab

3 papers 1,692

Datasets

Habitat Platform

MIDGARD

Subtasks

No subtasks available.

Most implemented papers

Habitat: A Platform for Embodied AI Research

Devi Parikh, Dhruv Batra, V. Koltun, Julian Straub, Jitendra Malik, M. Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Jia Liu•Mon Apr 01 2019

The comparison between learning and SLAM approaches from two recent works are revisited and evidence is found -- that learning outperforms SLAM if scaled to an order of magnitude more experience than previous investigations, and the first cross-dataset generalization experiments are conducted.

1692

Content

facebookresearch/habitat-challenge

3 papers 292

facebookresearch/habitat-api

2 papers 1,692

0

Paper Graph

Sim2Real Predictivity: Does Evaluation in Simulation Predict Real-World Performance?

Dhruv Batra, M. Savva, Abhishek Kadian, Erik Wijmans, Aaron Gokaslan, Alexander Clegg, Stefan Lee, S. Chernova, Joanne Truong•Thu Dec 12 2019

Does progress in simulation translate to progress on robots? If one method outperforms another in simulation, how likely is that trend to hold in reality on a robot? We examine this question for embodied PointGoal navigation – developing engineering tools and a research paradigm for evaluating a simulator by its sim2real predictivity. First, we develop Habitat-PyRobot Bridge (HaPy), a library for seamless execution of identical code on simulated agents and robots – transferring simulation-trained agents to a LoCoBot platform with a one-line code change. Second, we investigate the sim2real predictivity of Habitat-Sim M. Savva et al., for PointGoal navigation. We 3D-scan a physical lab space to create a virtualized replica, and run parallel tests of 9 different models in reality and simulation. We present a new metric called Sim-vs-Real Correlation Coefficient (SRCC) to quantify predictivity. We find that SRCC for Habitat as used for the CVPR19 challenge is low (0.18 for the success metric), suggesting that performance differences in this simulator-based challenge do not persist after physical deployment. This gap is largely due to AI agents learning to exploit simulator imperfections – abusing collision dynamics to ‘slide’ along walls, leading to shortcuts through otherwise non-navigable space. Naturally, such exploits do not work in the real world. Our experiments show that it is possible to tune simulation parameters to improve sim2real predictivity (e.g. improving SRCC$_{\text{Succ}}$ from 0.18 to 0.844) – increasing confidence that in-simulation comparisons will translate to deployed systems in reality.

248 0

Paper Graph

Learning to Explore using Active Neural SLAM

A. Gupta, R. Salakhutdinov, Saurabh Gupta, Devendra Singh Chaplot, Dhiraj Gandhi•Thu Apr 09 2020

This work presents a modular and hierarchical approach to learn policies for exploring 3D environments, called `Active Neural SLAM', which leverages the strengths of both classical and learning-based methods, by using analytical path planners with learned SLAM module, and global and local policies.

577 0

Paper Graph

Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI

Dhruv Batra, Angel X. Chang, M. Savva, Oleksandr Maksymets, Yili Zhao, Santhosh K. Ramakrishnan, Aaron Gokaslan, Erik Wijmans, Alexander Clegg, John Turner, Eric Undersander, Wojciech Galuba, Andrew Westbury•Wed Sep 15 2021

Habitat-Matterport 3D is a large-scale dataset of 1,000 building-scale 3D reconstructions from a diverse set of real-world locations that is `pareto optimal' in the following sense -- agents trained to perform PointGoal navigation on HM3D achieve the highest performance regardless of whether they are evaluated onHM3D, Gibson, or MP3D.

563 0

Paper Graph

Learning to Move with Affordance Maps

Deva Ramanan, William Qi, Saurabh Gupta, Ravi Teja Mullapudi•Tue Jan 07 2020

This paper designs an agent that learns to predict a spatial affordance map that elucidates what parts of a scene are navigable through active self-supervised experience gathering, and shows that learned affordance maps can be used to augment traditional approaches for both exploration and navigation, providing significant improvements in performance.

37 0

Paper Graph

Auxiliary Tasks Speed Up Learning PointGoal Navigation

Dhruv Batra, Abhishek Das, Erik Wijmans, Joel Ye•Wed Jul 08 2020

This work develops a method to significantly increase sample and time efficiency in learning PointNav using self-supervised auxiliary tasks (e.g. predicting the action taken between two egocentric observations, predicting the distance between two observations from a trajectory, etc.).

86 0

Paper Graph

Large Batch Simulation for Deep Reinforcement Learning

Dhruv Batra, V. Koltun, M. Savva, Erik Wijmans, Kayvon Fatahalian, Brennan Shacklett, Aleksei Petrenko•Thu Mar 11 2021

By combining batch simulation and DNN performance optimizations, it is demonstrated that PointGoal navigation agents can be trained in complex 3D environments on a single GPU in 1.5 days to 97% of the accuracy of agents trained on a prior state-of-the-art system using a 64-GPU cluster over three days.

28 0

Paper Graph

Out of the Box: Embodied Navigation in the Real World

Marcella Cornia, L. Baraldi, R. Cucchiara, Federico Landi, Roberto Bigazzi, Silvia Cascianelli•Tue May 11 2021

This work describes the architectural discrepancies that damage the Sim2Real adaptation ability of models trained on the Habitat simulator and proposes a novel solution tailored towards the deployment in real-world scenarios.

13 0

Paper Graph

Realistic PointGoal Navigation via Auxiliary Losses and Information Bottleneck

Dhruv Batra, Erik Wijmans, G. Grande•Thu Sep 16 2021

We propose a novel architecture and training paradigm for training realistic PointGoal Navigation -- navigating to a target coordinate in an unseen environment under actuation and sensor noise without access to ground-truth localization. Specifically, we find that the primary challenge under this setting is learning localization -- when stripped of idealized localization, agents fail to stop precisely at the goal despite reliably making progress towards it. To address this we introduce a set of auxiliary losses to help the agent learn localization. Further, we explore the idea of treating the precise location of the agent as privileged information -- it is unavailable during test time, however, it is available during training time in simulation. We grant the agent restricted access to ground-truth localization readings during training via an information bottleneck. Under this setting, the agent incurs a penalty for using this privileged information, encouraging the agent to only leverage this information when it is crucial to learning. This enables the agent to first learn navigation and then learn localization instead of conflating these two objectives in training. We evaluate our proposed method both in a semi-idealized (noiseless simulation without Compass+GPS) and realistic (addition of noisy simulation) settings. Specifically, our method outperforms existing baselines on the semi-idealized setting by 18\%/21\% SPL/Success and by 15\%/20\% SPL in the realistic setting. Our improved Success and SPL metrics indicate our agent's improved ability to accurately self-localize while maintaining a strong navigation policy. Our implementation can be found at https://github.com/NicoGrande/habitat-pointnav-via-ib.

1 0

Paper Graph

Adding a benchmark result helps the community track progress.