Direct-PoseNet: Absolute Pose Regression with Photometric Consistency (2021-04-08T00:00:00.000000Z)

TL;DR

This work designs a direct matching module that supplies a photometric supervision signal to refine the pose regression network via differentiable rendering and shows that the method can easily cope with additional unlabeled data without the need for external supervision.

Abstract

We present a relocalization pipeline, which combines an absolute pose regression (APR) network with a novel view synthesis based direct matching module, offering superior accuracy while maintaining low inference time. Our contribution is twofold: i) we design a direct matching module that supplies a photometric supervision signal to refine the pose regression network via differentiable rendering; ii) we show that our method can easily cope with additional unlabeled data without the need for external supervision such as traditional visual odometry or pose graph optimization. As a result, our method achieves state-of-the-art performance among all other single-image APR methods on the 7-Scenes benchmark and the LLFF dataset.

Authors

V. Prisacariu

5 papers

Zirui Wang

1 papers

Shuai Chen

1 papers

TL;DR

Abstract

Authors

References63 items

iNeRF: Inverting Neural Radiance Fields for Pose Estimation

NeRF++: Analyzing and Improving Neural Radiance Fields

NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections

Neural Sparse Voxel Fields

GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis

State of the Art on Neural Rendering

Visual Camera Re-Localization From RGB and RGB-D Images Using DSAC

Differentiable Volumetric Rendering: Learning Implicit 3D Representations Without 3D Supervision

Neural Point Cloud Rendering via Multi-Plane Projection

PyTorch: An Imperative Style, High-Performance Deep Learning Library

InLoc: Indoor Visual Localization with Dense Matching and View Synthesis

Semi-Supervised Learning

Introduction to Camera Pose Estimation with Deep Learning

Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations

Neural-Guided RANSAC: Learning Where to Sample Model Hypotheses

Deferred neural rendering

Neural Rerendering in the Wild

Understanding the Limitations of CNN-Based Absolute Camera Pose Regression

DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation

Occupancy Networks: Learning 3D Reconstruction in Function Space

DeepVoxels: Learning Persistent 3D Feature Embeddings

RelocNet: Continuous Metric Learning Relocalisation Using Neural Nets

VLocNet++: Deep Multitask Learning for Semantic Visual Localization and Odometry

Deep Auxiliary Learning for Visual Localization and Odometry

MobileNetV2: Inverted Residuals and Linear Bottlenecks

Geometry-Aware Learning of Maps for Camera Localization

Learning Less is More - 6D Camera Localization via 3D Surface Regression

Deep regression for monocular camera-based 6-DoF global localization in outdoor environments

Efficient & Effective Prioritized Matching for Large-Scale Image-Based Localization

Delving deeper into convolutional neural networks for camera relocalization

Image-Based Localization Using Hourglass Networks

Geometric Loss Functions for Camera Pose Regression with Deep Learning

Image-Based Localization Using LSTMs for Structured Feature Correlation

DSAC — Differentiable RANSAC for Camera Localization

Modelling uncertainty in deep learning for camera relocalization

PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Adam: A Method for Stochastic Optimization

REMODE: Probabilistic, monocular dense reconstruction in real time

SVO: Fast semi-direct monocular visual odometry

Real-time RGB-D camera relocalization

Semi-dense Visual Odometry for a Monocular Camera

USAC: A Universal Framework for Random Sample Consensus

Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images

Improving Image-Based Localization by Active Correspondence Search

DTAM: Dense tracking and mapping in real-time

Fast image-based localization using direct 2D-to-3D matching

Optimal Randomized RANSAC

Segmentation by transduction

Parallel Tracking and Mapping for Small AR Workspaces

MonoSLAM: Real-Time Single Camera SLAM

Real-time 3D motion and structure of point features: a front-end system for vision-based control and interaction

Light field rendering

The lumigraph

View Interpolation for Image Synthesis

NeRF-: Neural Radiance Fields Without Known Camera Parameters

PyTorchNeRF: a PyTorch implementation of NeRF

Neural volumes: Learning dynamic renderable volumes from images

Local light ﬁeld fusion: Practical view synthesis with prescriptive sampling guidelines

DSO: Direct sparse odometry

All About Direct Methods

Representing Scenes as Neural Radiance Fields for View Synthesis

Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names