Beyond Controlled Environments: 3D Camera Re-Localization in Changing Indoor Scenes (2020-08-05T00:00:00.000000Z)

TL;DR

This paper adapts 3RScan - a recently introduced indoor RGB-D dataset designed for object instance re-localization - to create RIO10, a new long-term camera re- localization benchmark focused on indoor scenes and explores how state-of-the-art cameraRe-localizers perform according to these metrics.

Abstract

Long-term camera re-localization is an important task with numerous computer vision and robotics applications. Whilst various outdoor benchmarks exist that target lighting, weather and seasonal changes, far less attention has been paid to appearance changes that occur indoors. This has led to a mismatch between popular indoor benchmarks, which focus on static scenes, and indoor environments that are of interest for many real-world applications. In this paper, we adapt 3RScan - a recently introduced indoor RGB-D dataset designed for object instance re-localization - to create RIO10, a new long-term camera re-localization benchmark focused on indoor scenes. We propose new metrics for evaluating camera re-localization and explore how state-of-the-art camera re-localizers perform according to these metrics. We also examine in detail how different types of scene change affect the performance of different methods, based on novel ways of detecting such changes in a given RGB-D frame. Our results clearly show that long-term indoor re-localization is an unsolved problem. Our benchmark and tools are publicly available at this http URL

Authors

S. Golodetz

6 papers

Federico Tombari

15 papers

Johanna Wald

4 papers

TL;DR

Abstract

Authors

References112 items

Visual Camera Re-Localization From RGB and RGB-D Images Using DSAC

InLoc: Indoor Visual Localization with Dense Matching and View Synthesis

SANet: Scene Agnostic Network for Camera Localization

SIR-Net: Scene-Independent End-to-End Trainable Visual Relocalizer

Fine-Grained Segmentation Networks: Self-Supervised Segmentation for Improved Long-Term Visual Localization

RIO: 3D Object Instance Re-Localization in Changing Indoor Environments

Is This the Right Place? Geometric-Semantic Pose Verification for Indoor Visual Localization

Expert Sample Consensus Applied to Camera Re-Localization

To Learn or Not to Learn: Visual Localization from Essential Matrices

Sparse-to-Dense Hypercolumn Matching for Long-Term Visual Localization

ELF: Embedded Localisation of Features in Pre-Trained CNN

Let's Take This Online: Adapting Scene Coordinate Regression Network Predictions for Online RGB-D Camera Relocalisation

Neural-Guided RANSAC: Learning Where to Sample Model Hypotheses

D2-Net: A Trainable CNN for Joint Description and Detection of Local Features

Characterizing Visual Localization and Mapping Datasets

Semantic Mapping for View-Invariant Relocalization

BIM-PoseNet: Indoor camera localisation using a 3D indoor model and deep learning from synthetic images

Understanding the Limitations of CNN-Based Absolute Camera Pose Regression

Adversarial Networks for Camera Pose Regression and Refinement

Relative Geometry-Aware Siamese Neural Network for 6DOF Camera Relocalization

From Coarse to Fine: Robust Hierarchical Localization at Large Scale

Real-Time RGB-D Camera Pose Estimation in Novel Scenes Using a Relocalisation Cascade

xyzNet: Towards Machine Learning Camera Relocalization by Using a Scene Coordinate Prediction Network

Night-to-Day Image Translation for Retrieval-based Localization

RelocNet: Continuous Metric Learning Relocalisation Using Neural Nets

Semantic Match Consistency for Long-Term Visual Localization

InteriorNet: Mega-scale Multi-sensor Photo-realistic Indoor Scenes Dataset

Scene Coordinate Regression with Angle-Based Reprojection Loss for Camera Relocalization

Benchmarking 6DOF Outdoor Visual Localization in Changing Conditions

Structure from motion using dense CNN features with keypoint relocalization

VLocNet++: Deep Multitask Learning for Semantic Visual Localization and Odometry

Local Descriptors Optimized for Average Precision

Adversarial Training for Adverse Conditions: Robust Metric Localisation using Appearance Transfer

Deep Auxiliary Learning for Visual Localization and Odometry

Full-Frame Scene Coordinate Regression for Image-Based Localization

24/7 Place Recognition by View Synthesis

Collaborative Large-Scale Dense 3D Reconstruction with Online Inter-Agent Pose Optimisation

SuperPoint: Self-Supervised Interest Point Detection and Description

Semantic Visual Localization

Geometry-Aware Learning of Maps for Camera Localization

Learning Less is More - 6D Camera Localization via 3D Surface Regression

Exploiting Points and Lines in Regression Forests for RGB-D Camera Relocalization

Long-Term 3D Localization and Pose from Semantic Labellings

[POSTER] Decision Forest For Efficient and Robust Camera Relocalization

Matterport3D: Learning from RGB-D Data in Indoor Environments

Simultaneous Localization and Mapping: A Survey of Current Trends in Autonomous Driving

Backtracking regression forests for accurate camera relocalization

Efficient & Effective Prioritized Matching for Large-Scale Image-Based Localization

Camera Relocalization by Computing Pairwise Relative Poses Using Convolutional Neural Network

Delving deeper into convolutional neural networks for camera relocalization

Image-Based Localization Using Hourglass Networks

Geometric Loss Functions for Camera Pose Regression with Deep Learning

VidLoc: A Deep Spatio-Temporal Model for 6-DoF Video-Clip Relocalization

ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes

On-the-Fly Adaptation of Regression Forests for Online Camera Relocalisation

Joint 2D-3D-Semantic Data for Indoor Scene Understanding

’Scape

1 year, 1000 km: The Oxford RobotCar dataset

Exploiting 2D Floorplan for Building-Scale Panorama RGBD Alignment

Image-Based Localization Using LSTMs for Structured Feature Correlation

DSAC — Differentiable RANSAC for Camera Localization

Incremental image set querying based localization

SceneNN: A Scene Meshes Dataset with aNNotations

Random forests versus Neural Networks — What's best for camera localization?

University of Michigan North Campus long-term vision and lidar dataset

Fast and scalable structure-from-motion based localization for high-precision mobile augmented reality systems

Efficient Intersection of Three Quadrics and Applications in Computer Vision

Uncertainty-Driven 6D Pose Estimation of Objects and Scenes from a Single RGB Image

Modelling and correcting for the impact of the gait cycle on touch screen typing accuracy

Learning to Navigate the Energy Landscape

A Fast 3D Indoor-Localization Approach Based on Video Queries

Localization from semantic observations via the matrix permanent

NetVLAD: CNN Architecture for Weakly Supervised Place Recognition

SemanticPaint

Marker-Less AR in the Hybrid Room Using Equipment Detection for Camera Relocalization

Modelling uncertainty in deep learning for camera relocalization