ContextDesc: Local Descriptor Augmentation With Cross-Modality Context (2019-04-08T00:00:00.000000Z)

TL;DR

This paper proposes a unified learning framework that leverages and aggregates the cross-modality contextual information, including visual context from high-level image representation, and geometric context from 2D keypoint distribution, and proposes an effective N-pair loss that eschews the empirical hyper-parameter search and improves the convergence.

Abstract

Most existing studies on learning local features focus on the patch-based descriptions of individual keypoints, whereas neglecting the spatial relations established from their keypoint locations. In this paper, we go beyond the local detail representation by introducing context awareness to augment off-the-shelf local feature descriptors. Specifically, we propose a unified learning framework that leverages and aggregates the cross-modality contextual information, including (i) visual context from high-level image representation, and (ii) geometric context from 2D keypoint distribution. Moreover, we propose an effective N-pair loss that eschews the empirical hyper-parameter search and improves the convergence. The proposed augmentation scheme is lightweight compared with the raw local feature description, meanwhile improves remarkably on several large-scale benchmarks with diversified scenes, which demonstrates both strong practicality and generalization ability in geometric matching applications.

Authors

Yao Yao

6 papers

Zixin Luo

6 papers

Shiwei Li

5 papers

TL;DR

Abstract

Authors

References57 items

Distributed Very Large Scale Bundle Adjustment by Global Camera Consensus

Matchable Image Retrieval by Learning from Surface Reconstruction

Large scale evaluation of local image feature detectors on homography datasets

GeoDesc: Learning Local Descriptors by Integrating Geometry Constraints

Learning and Matching Multi-View Descriptors for Registration of Point Clouds

Learning Deep Descriptors with Scale-Aware Triplet Networks

Very Large-Scale Global SfM by Distributed Motion Averaging

Learning to Detect Features in Texture Images

Local Descriptors Optimized for Average Precision

PPFNet: Global Context Aware Local Features for Robust 3D Point Matching

Fix your classifier: the marginal value of training the last weight layer

End-to-End Weakly-Supervised Semantic Alignment

Repeatability Is Not Enough: Learning Affine Regions via Discriminability

Learning to Find Good Correspondences

Progressive Large Scale-Invariant Image Matching in Scale Space

Learning Spread-Out Local Feature Descriptors

L2-Net: Deep Learning of Discriminative Patch Descriptor in Euclidean Space

Comparative Evaluation of Hand-Crafted and Learned Local Features

Deep Semantic Feature Matching

PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space

Working hard to know your neighbor's margins: Local descriptor learning loss

HPatches: A Benchmark and Evaluation of Handcrafted and Learned Local Descriptors

Large-Scale Image Retrieval with Attentive Deep Local Features

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation

Quad-Networks: Unsupervised Learning to Rank for Interest Point Detection

Graph-Based Consistent Matching for Structure-from-Motion

Structure-from-Motion Revisited

Deep Image Homography Estimation

CNN Image Retrieval Learns from BoW: Unsupervised Fine-Tuning with Hard Examples

LIFT: Learned Invariant Feature Transform

Identity Mappings in Deep Residual Networks

PN-Net: Conjoined Triple Deep Network for Learning Local Image Descriptors

Learning Local Image Descriptors with Deep Siamese and Triplet Convolutional Networks by Minimizing Global Loss Functions

Deep Residual Learning for Image Recognition

Dual-Feature Warping-Based Motion Model Estimation

Discriminative Learning of Deep Convolutional Feature Point Descriptors

Particular object retrieval with integral max-pooling of CNN activations

Learning to Assign Orientations to Feature Points

MatchNet: Unifying feature and metric learning for patch-based matching

Spatial Transformer Networks

Learning to compare image patches via convolutional neural networks

YFCC100M

Domain-size pooling in local descriptors: DSP-SIFT

Matching Features Correctly through Semantic Understanding

Robust Global Translations with 1DSfM

Local Readjustment for High-Resolution 3D Reconstruction

Predicting Matchability

SUN3D: A Database of Big Spaces Reconstructed Using SfM and Object Labels

Comparative Evaluation of Binary Features

On benchmarking camera calibration and multi-view stereo for high resolution imagery

Lost in quantization: Improving particular object retrieval in large scale image databases

Automatic Panoramic Image Stitching using Invariant Features

Object retrieval with large vocabularies and fast spatial matching

Scalable Recognition with a Vocabulary Tree

Robust wide-baseline stereo from maximally stable extremal regions

Learning local feature descriptors with triplets and shallow convolutional neural networks

Distinctive Image Features from Scale-Invariant Keypoints Abstract by Matthijs Dorst Based on the paper by

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names