SuperGlue: Learning Feature Matching With Graph Neural Networks (2019-11-26T00:00:00.000000Z)

TL;DR

SuperGlue is introduced, a neural network that matches two sets of local features by jointly finding correspondences and rejecting non-matchable points and introduces a flexible context aggregation mechanism based on attention, enabling SuperGlue to reason about the underlying 3D scene and feature assignments jointly.

Abstract

This paper introduces SuperGlue, a neural network that matches two sets of local features by jointly finding correspondences and rejecting non-matchable points. Assignments are estimated by solving a differentiable optimal transport problem, whose costs are predicted by a graph neural network. We introduce a flexible context aggregation mechanism based on attention, enabling SuperGlue to reason about the underlying 3D scene and feature assignments jointly. Compared to traditional, hand-designed heuristics, our technique learns priors over geometric transformations and regularities of the 3D world through end-to-end training from image pairs. SuperGlue outperforms other learned approaches and achieves state-of-the-art results on the task of pose estimation in challenging real-world indoor and outdoor environments. The proposed method performs matching in real-time on a modern GPU and can be readily integrated into modern SfM or SLAM systems. The code and trained weights are publicly available at github.com/magicleap/SuperGluePretrainedNetwork.

Authors

Daniel DeTone

4 papers

Tomasz Malisiewicz

3 papers

Andrew Rabinovich

6 papers

TL;DR

Abstract

Authors

References74 items

UR2KiD: Unifying Retrieval, Keypoint Detection, and Keypoint Description without Local Correspondence Supervision

Dual Graph Convolutional Network for Semantic Segmentation

Beyond Cartesian Representations for Local Descriptors

Learning Two-View Correspondences and Geometry Using Order-Aware Network

R2D2: Repeatable and Reliable Detector and Descriptor

Neural-Guided RANSAC: Learning Where to Sample Model Hypotheses

Deep Closest Point: Learning Representations for Point Cloud Registration

Graph Matching Networks for Learning the Similarity of Graph Structured Objects

ContextDesc: Local Descriptor Augmentation With Cross-Modality Context

From Coarse to Fine: Robust Hierarchical Localization at Large Scale

Self-Improving Visual Odometry

Neighbourhood Consensus Networks

Set Transformer

Deep Fundamental Matrix Estimation

SConE: Siamese Constellation Embedding Descriptor for Image Matching

Relational inductive biases, deep learning, and graph networks

Benchmarking 6DOF Outdoor Visual Localization in Changing Conditions

LF-Net: Learning Local Features from Images

MegaDepth: Learning Single-View Depth Prediction from Internet Photos

Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking

Computational Optimal Transport

PPFNet: Global Context Aware Local Features for Robust 3D Point Matching

Dynamic Graph CNN for Learning on Point Clouds

SuperPoint: Self-Supervised Interest Point Detection and Description

Non-local Neural Networks

Learning to Find Good Correspondences

Graph Attention Networks

Automatic differentiation in PyTorch

GMS: Grid-Based Motion Statistics for Fast, Ultra-Robust Feature Correspondence

Attention is All you Need

PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space

Convolutional Sequence to Sequence Learning

HPatches: A Benchmark and Evaluation of Handcrafted and Learned Local Descriptors

Neural Message Passing for Quantum Chemistry

Deep Sets

ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation

Pixelwise View Selection for Unstructured Multi-View Stereo

Instance Normalization: The Missing Ingredient for Fast Stylization

Structure-from-Motion Revisited

Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age

Deep Image Homography Estimation

BundleFusion

LIFT: Learned Invariant Feature Transform

Adam: A Method for Stochastic Optimization

SUN3D: A Database of Big Spaces Reconstructed Using SfM and Object Labels

Sinkhorn Distances: Lightspeed Computation of Optimal Transport

Growing multiplex networks

Three things everyone should know to improve object retrieval

ORB: An efficient alternative to SIFT or SURF

Image retrieval with geometry-preserving visual phrases

Community Structure in Time-Dependent, Multiscale, and Multiplex Networks

SCRAMSAC: Improving RANSAC's efficiency with a spatial consistency filter

Optimal Transport: Old and New

A Comparative Analysis of RANSAC Techniques Leading to Adaptive Real-Time Random Sample Consensus

Feature Correspondence Via Graph Matching: Models and Global Optimization

Learning Graph Matching

Matching Local Self-Similarities across Images and Videos

A survey for the quadratic assignment problem

Shape matching and object recognition using low distortion correspondences

Distinctive Image Features from Scale-Invariant Keypoints

Concerning nonnegative matrices and doubly stochastic matrices

D2-Net: A trainable CNN for joint detection and description of local features

Phototourism Challenge, CVPR 2019 Image Matching Workshop

YFCC100M: The new data in multimedia research

A Survey of the Quadratic Assignment Problem

Image Retrieval for Image-Based Localization Revisited

Multiple View Geometry in Computer Vision

Wide Baseline Stereo Matching based on Local, Affinely Invariant Regions

ALGORITHMS FOR THE ASSIGNMENT AND TRANSIORTATION tROBLEMS*

Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography

Contextual cueing of visual attention

Ieee Transactions on Pattern Analysis and Machine Intelligence 1 Efficient Sequential Correspondence Selection by Cosegmentation

ORB: an efﬁcient alternative to SIFT or SURF

Field of Study

Journal Information