Visual Odometry Revisited: What Should Be Learnt?

Published in

IEEE International Conference on Robotics and A...(2019)

External Links:

Generate Graph DownloadPDF

TL;DR

This work revisit the basics of VO and explore the right way for integrating deep learning with epipolar geometry and Perspective-n-Point method and design a simple but robust frame-to-frame VO algorithm (DF-VO) which outperforms pure deep learning-based and geometry-based methods.

Abstract

In this work we present a monocular visual odometry (VO) algorithm which leverages geometry-based methods and deep learning. Most existing VO/SLAM systems with superior performance are based on geometry and have to be carefully designed for different application scenarios. Moreover, most monocular systems suffer from scale-drift issue. Some recent deep learning works learn VO in an end-to-end manner but the performance of these deep systems is still not comparable to geometry-based methods. In this work, we revisit the basics of VO and explore the right way for integrating deep learning with epipolar geometry and Perspective-n-Point (PnP) method. Specifically, we train two convolutional neural networks (CNNs) for estimating single-view depths and two-view optical flows as intermediate outputs. With the deep predictions, we design a simple but robust frame-to-frame VO algorithm (DF-VO) which outperforms pure deep learning-based and geometry-based methods. More importantly, our system does not suffer from the scale-drift issue being aided by a scale consistent single-view depth CNN. Extensive experiments on KITTI dataset shows the robustness of our system and a detailed ablation study shows the effect of different factors in our system. Code is available at here: DF-VO.

Authors

I. Reid

18 papers

Jiawang Bian

4 papers

Huangying Zhan

3 papers

References64 items

An Evaluation of Feature Matchers for Fundamental Matrix Estimation

Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video

Pose Graph optimization for Unsupervised Monocular Visual Odometry

Self-supervised Learning for Single View Depth and Surface Normal Estimation

CNN-SVO: Improving the Mapping in Semi-Direct Visual Odometry Using Single-Image Depth Prediction

Visual Odometry Revisited: What Should Be Learnt?

Published in

IEEE International Conference on Robotics and A...(2019)

External Links:

Generate Graph DownloadPDF

TL;DR

Abstract

Authors

I. Reid

18 papers

Jiawang Bian

4 papers

Huangying Zhan

3 papers

References64 items

An Evaluation of Feature Matchers for Fundamental Matrix Estimation

Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video

Pose Graph optimization for Unsupervised Monocular Visual Odometry

Self-supervised Learning for Single View Depth and Surface Normal Estimation

CNN-SVO: Improving the Mapping in Semi-Direct Visual Odometry Using Single-Image Depth Prediction

C. Weerasekera

2 papers

Real-Time Joint Semantic Segmentation and Depth Estimation Using Asymmetric Annotations

DeepTAM: Deep Tracking and Mapping

ENG: End-to-end Neural Geometry for Robust Depth and Pose Estimation using CNNs

Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometry

Deep Ordinal Regression Network for Monocular Depth Estimation

Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentation

Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstruction

GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose

Learning Less is More - 6D Camera Localization via 3D Surface Regression

UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Loss

Automatic differentiation in PyTorch

UnDeepVO: Monocular Visual Odometry Through Unsupervised Deep Learning

PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume

GMS: Grid-Based Motion Statistics for Fast, Ultra-Robust Feature Correspondence

Unsupervised Learning of Depth and Ego-Motion from Video

CNN-SLAM: Real-Time Dense Monocular SLAM with Learned Depth Prediction

SVO: Semidirect Visual Odometry for Monocular and Multicamera Systems

What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?

DeepVO: Towards end-to-end visual odometry with deep Recurrent Convolutional Neural Networks

DeMoN: Depth and Motion Network for Learning Monocular Stereo

FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks

DSAC — Differentiable RANSAC for Camera Localization

ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras

Unsupervised Monocular Depth Estimation with Left-Right Consistency

Direct Sparse Odometry

Deeper Depth Prediction with Fully Convolutional Residual Networks

Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue

PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization

U-Net: Convolutional Networks for Biomedical Image Segmentation

Learning to See by Moving

FlowNet: Learning Optical Flow with Convolutional Networks

Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields

ORB-SLAM: A Versatile and Accurate Monocular SLAM System

Adam: A Method for Stochastic Optimization

Deep convolutional neural fields for depth estimation from a single image

SVO: Fast semi-direct monocular visual odometry

LSD-SLAM: Large-Scale Direct Monocular SLAM

Depth Map Prediction from a Single Image using a Multi-Scale Deep Network

Vision meets robotics: The KITTI dataset

Are we ready for autonomous driving? The KITTI vision benchmark suite

DTAM: Dense tracking and mapping in real-time

ORB: An efficient alternative to SIFT or SURF

StereoScan: Dense 3d reconstruction in real-time

Parallel Tracking and Mapping for Small AR Workspaces

Distinctive Image Features from Scale-Invariant Keypoints

An efficient solution to the five-point relative pose problem

Determining the Epipolar Geometry and its Uncertainty: A Review

In defence of the 8-point algorithm

Least-Squares Estimation of Transformation Parameters Between Two Point Patterns

Automatic Differentiation: Techniques and Applications

The interpretation of structure from motion

Digging into self-supervised monocular depth prediction

Supplementary Material for LiteFlowNet: A Lightweight Convolutional Neural Network for Optical Flow Estimation

International Journal of Computer Vision c ○ 2007 Springer Science + Business Media, LLC. Manufactured in the United States. DOI: 10.1007/s11263-007-0046-z Computer Vision on Mars

Multiple View Geometry in Computer Vision (2nd ed)

Multiple View Geometry in Computer Vision

The interpretation of visual motion

Visual Odometry Part I: The First 30 Years and Fundamentals

Field of Study

Computer Science

Journal Information

Name

2020 IEEE International Conference on Robotics and Automation (ICRA)

Page

3291-3297

Venue Information

Name

IEEE International Conference on Robotics and Automation

Type

conference

URL

http://ieeexplore.ieee.org/xpl/conhome.jsp?punumber=1000639

Alternate Names

International Conference on Robotics and Automation
Int Conf Robot Autom
ICRA
IEEE Int Conf Robot Autom