Smooth-AP: Smoothing the Path Towards Large-Scale Image Retrieval (2020-07-23T00:00:00.000000Z)

TL;DR

Smooth-AP is a plug-and-play objective function that allows for end-to-end training of deep networks with a simple and elegant implementation and improves the performance over the state-of-the-art, especially for larger-scale datasets, thus demonstrating the effectiveness and scalability of Smooth-AP to real-world scenarios.

Abstract

Optimising a ranking-based metric, such as Average Precision (AP), is notoriously challenging due to the fact that it is non-differentiable, and hence cannot be optimised directly using gradient-descent methods. To this end, we introduce an objective that optimises instead a smoothed approximation of AP, coined Smooth-AP. Smooth-AP is a plug-and-play objective function that allows for end-to-end training of deep networks with a simple and elegant implementation. We also present an analysis for why directly optimising the ranking based metric of AP offers benefits over other deep metric learning losses. We apply Smooth-AP to standard retrieval benchmarks: Stanford Online products and VehicleID, and also evaluate on larger-scale datasets: INaturalist for fine-grained category retrieval, and VGGFace2 and IJB-C for face retrieval. In all cases, we improve the performance over the state-of-the-art, especially for larger-scale datasets, thus demonstrating the effectiveness and scalability of Smooth-AP to real-world scenarios.

Authors

Andrew Zisserman

62 papers

Weidi Xie

19 papers

Vicky Kalogeiton

3 papers

TL;DR

Abstract

Authors

References75 items

SoftSort: A Continuous Relaxation for the argsort Operator

Smoothing

Cross-Batch Memory for Embedding Learning

Optimizing Rank-Based Metrics With Blackbox Differentiation

Differentiation of Blackbox Combinatorial Solvers

Sampling Wisely: Deep Image Embedding by Top-K Precision Optimization

MIC: Mining Interclass Characteristics for Improved Metric Learning

SoftTriple Loss: Deep Metric Learning Without Triplet Sampling

AutoCorrect: Deep Inductive Alignment of Noisy Geometric Annotations

Learning With Average Precision: Training Image Retrieval With a Listwise Loss

Deep Metric Learning to Rank

Divide and Conquer the Embedding Space for Metric Learning

Stochastic Class-Based Hard Example Mining for Deep Metric Learning

Multi-Similarity Loss With General Pair Weighting for Deep Metric Learning

Towards Accurate One-Stage Object Detection With AP-Loss

SoDeep: A Sorting Deep Net to Learn Ranking Loss Surrogates

Ranked List Loss for Deep Metric Learning

Deep Metric Learning with Hierarchical Triplet Loss

Deep Randomized Ensembles for Metric Learning

Deep Adversarial Metric Learning

Learning Globally Optimized Object Detector via Policy Gradient

Local Descriptors Optimized for Average Precision

Attention-based Ensemble for Deep Metric Learning

IARPA Janus Benchmark - C: Face Dataset and Protocol

ArcFace: Additive Angular Margin Loss for Deep Face Recognition

VGGFace2: A Dataset for Recognising Faces across Pose and Age

BIER — Boosting Independent Embeddings Robustly

Squeeze-and-Excitation Networks

Deep Metric Learning with Angular Loss

The iNaturalist Species Classification and Detection Dataset

Deep Spectral Clustering Learning

Sampling Matters in Deep Embedding Learning

Hashing as Tie-Aware Learning to Rank

SphereFace: Deep Hypersphere Embedding for Face Recognition

Smart Mining for Deep Metric Learning

In Defense of the Triplet Loss for Person Re-Identification

No Fuss Distance Metric Learning Using Proxies

Deep Metric Learning via Facility Location

Hard-Aware Deeply Cascaded Embedding

Learning Deep Embeddings with Histogram Loss

Cross-batch Reference Learning for Deep Classification and Retrieval

End-to-End Training of Object Class Detectors for Mean Average Precision

Efficient Optimization for Rank-Based Loss Functions

CNN Image Retrieval Learns from BoW: Unsupervised Fine-Tuning with Hard Examples

Deep Image Retrieval: Learning Global Representations for Image Search

Deep Residual Learning for Image Recognition

NetVLAD: CNN Architecture for Weakly Supervised Place Recognition

Deep Metric Learning via Lifted Structured Feature Embedding

FaceNet: A unified embedding for face recognition and clustering

Adam: A Method for Stochastic Optimization

ImageNet Large Scale Visual Recognition Challenge

A maximal figure-of-merit learning approach to maximizing mean average precision with deep neural network based classifiers

Learning Fine-Grained Image Similarity with Deep Ranking

Neural Codes for Image Retrieval

All About VLAD

Aggregating Local Image Descriptors into Compact Codes

The Caltech-UCSD Birds-200-2011 Dataset

Total recall II: Query expansion revisited

A general approximation framework for direct optimization of information retrieval measures

Metric Learning to Rank

Large-scale image retrieval with compressed Fisher vectors

Aggregating local descriptors into a compact image representation

Hamming Embedding and Weak Geometric Consistency for Large Scale Image Search

Learning to rank with SoftRank and Gaussian processes

Introduction to Information Retrieval

SoftRank: optimizing non-smooth rank metrics

A support vector method for optimizing average precision

Learning to rank: from pairwise approach to listwise approach

Object retrieval with large vocabularies and fast spatial matching

Learning to Rank with Nonsmooth Cost Functions

Distance Metric Learning for Large Margin Nearest Neighbor Classification

Learning a similarity metric discriminatively, with application to face verification

Video Google: a text retrieval approach to object matching in videos

Product Quantization for Nearest Neighbor Search

Large margin optimization of ranking measures

Field of Study