ALIKE: Accurate and Lightweight Keypoint Detection and Descriptor Extraction (2021-12-06T00:00:00.000000Z)

TL;DR

A partially differentiable keypoint detection module is presented, which outputs accurate sub-pixel keypoints and the reprojection loss is then proposed to directly optimize these sub- pixel keypoints, and the dispersity peak loss is presented for accurate keypoints regularization.

Abstract

Existing methods detect the keypoints in a non-differentiable way, therefore they can not directly optimize the position of keypoints through back-propagation. To address this issue, we present a partially differentiable keypoint detection module, which outputs accurate sub-pixel keypoints. The reprojection loss is then proposed to directly optimize these sub-pixel keypoints, and the dispersity peak loss is presented for accurate keypoints regularization. We also extract the descriptors in a sub-pixel way, and they are trained with the stable neural reprojection error loss. Moreover, a lightweight network is designed for keypoint detection and descriptor extraction, which can run at 95 frames per second for $640\times 480$ images on a commercial GPU. On homography estimation, camera pose estimation, and visual (re-)localization tasks, the proposed method achieves equivalent performance with the state-of-the-art approaches, while greatly reduces the inference time.

Authors

Xingming Wu

3 papers

Peter C. Y. Chen

2 papers

Xiaoming Zhao

1 papers

TL;DR

Abstract

Authors

References52 items

Removing the Bias of Integral Pose Regression

Neural Reprojection Error: Merging Feature Learning and Camera Pose Estimation

Image Matching from Handcrafted to Deep Features: A Survey

DISK: Learning local features with policy gradient

SEKD: Self-Evolving Keypoint Detection and Description

D2D: Keypoint Extraction with Describe to Detect Approach

VPR-Bench: An Open-Source Visual Place Recognition Evaluation Framework with Quantifiable Viewpoint and Appearance Change

HDD-Net: Hybrid Detector Descriptor with Mutual Interactive Learning

Learning Feature Descriptors using Camera Pose Supervision

S2DNet: Learning Accurate Correspondences for Sparse-to-Dense Feature Matching

ASLFeat: Learning Local Features of Accurate Shape and Localization

Image Matching Across Wide Baselines: From Paper to Practice

UR2KiD: Unifying Retrieval, Keypoint Detection, and Keypoint Description without Local Correspondence Supervision

Reinforced Feature Points: Optimizing Feature Detection and Description for a High-Level Task

Sparse-to-Dense Hypercolumn Matching for Long-Term Visual Localization

R2D2: Repeatable and Reliable Detector and Descriptor

D2-Net: A Trainable CNN for Joint Description and Detection of Local Features

SOSNet: Second Order Similarity Regularization for Local Descriptor Learning

Key.Net: Keypoint Detection by Handcrafted and Learned CNN Filters

Learning-Based Tone Mapping Operator for Efficient Image Matching

Dual-mode Dynamic Window Approach to Robot Navigation with Convergence Guarantees

Benchmarking 6DOF Outdoor Visual Localization in Changing Conditions

LF-Net: Learning Local Features from Images

Local Descriptors Optimized for Average Precision

MegaDepth: Learning Single-View Depth Prediction from Internet Photos

SuperPoint: Self-Supervised Interest Point Detection and Description

Integral Human Pose Regression

Intelligent Detail Enhancement for Exposure Fusion

L2-Net: Deep Learning of Discriminative Patch Descriptor in Euclidean Space

Working hard to know your neighbor's margins: Local descriptor learning loss

HPatches: A Benchmark and Evaluation of Handcrafted and Learned Local Descriptors

Quad-Networks: Unsupervised Learning to Rank for Interest Point Detection

ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras

Keypoint Detection in RGBD Images Based on an Anisotropic Scale Space

Structure-from-Motion Revisited

LIFT: Learned Invariant Feature Transform

Deep Residual Learning for Image Recognition

MatchNet: Unifying feature and metric learning for patch-based matching

Adam: A Method for Stochastic Optimization

TILDE: A Temporally Invariant Learned DEtector

Hybrid Patching for a Sequence of Differently Exposed Images With Moving Objects

ORB: An efficient alternative to SIFT or SURF

Rectified Linear Units Improve Restricted Boltzmann Machines

Gradient descent optimization of smoothed information retrieval metrics

Shot Change Detection via Local Keypoint Matching

Distinctive Image Features from Scale-Invariant Keypoints

A performance evaluation of local descriptors

Robust and Efficient RGB-D SLAM in Dynamic Environments

MLIFeat: Multi-level Information Fusion Based Deep Local Features

S2DNet: Learning Image Features for Accurate Sparse-to-Dense Matching

Learning local feature descriptors with triplets and shallow convolutional neural networks

The differentiable keypoint detection and descriptor sampling ﬁrst detects the sub-pixel keypoints from the score map S

Field of Study

Journal Information

Name

Page

Volume

Venue Information

Name

Type

URL

Alternate Names