The 2021 Image Similarity Dataset and Challenge (2021-06-17T00:00:00.000000Z)

TL;DR

A new benchmark for large-scale image similarity detection using a variety of image transformations such as automated transformations, hand-crafted image edits and machine-learning based manipulations to promote image copy detection as an important and challenging computer vision task and refresh the state of the art.

Abstract

This paper introduces a new benchmark for large-scale image similarity detection. This benchmark is used for the Image Similarity Challenge at NeurIPS'21 (ISC2021). The goal is to determine whether a query image is a modified copy of any image in a reference corpus of size 1~million. The benchmark features a variety of image transformations such as automated transformations, hand-crafted image edits and machine-learning based manipulations. This mimics real-life cases appearing in social media, for example for integrity-related problems dealing with misinformation and objectionable content. The strength of the image manipulations, and therefore the difficulty of the benchmark, is calibrated according to the performance of a set of baseline approaches. Both the query and reference set contain a majority of"distractor"images that do not match, which corresponds to a real-life needle-in-haystack setting, and the evaluation metric reflects that. We expect the DISC21 benchmark to promote image copy detection as an important and challenging computer vision task and refresh the state of the art. Code and data are available at https://github.com/facebookresearch/isc2021

Authors

Matthijs Douze

10 papers

Giorgos Tolias

11 papers

Ondřej Chum

7 papers

TL;DR

Abstract

Authors

References73 items

ScaleNet: An Unsupervised Representation Learning Method for Limited Information

A Self-Supervised Descriptor for Image Copy Detection

AugLy: Data Augmentations for Robustness

Watermarking Images in Self-Supervised Latent Spaces

Emerging Properties in Self-Supervised Vision Transformers

Preserving integrity in online social networks

Learning and aggregating deep local descriptors for instance-level recognition

The DeepFake Detection Challenge Dataset

CIAGAN: Conditional Identity Anonymization Generative Adversarial Networks

Google Landmarks Dataset v2 – A Large-Scale Benchmark for Instance-Level Recognition and Retrieval

A Simple Framework for Contrastive Learning of Visual Representations

Momentum Contrast for Unsupervised Visual Representation Learning

Targeted Mismatch Adversarial Attack: Query With a Flower to Retrieve the Tower

ViSiL: Fine-Grained Spatio-Temporal Video Similarity Learning

Detecting Photoshopped Faces by Scripting Photoshop

Local Features and Visual Words Emerge in Activations

MultiGrain: a unified image embedding for classes and instances

A Simple Explanation for the Existence of Adversarial Examples with Small Hamming Distance

FIVR: Fine-Grained Incident Video Retrieval

Learning spatial-temporal features for video copy detection by the combination of CNN and RNN

Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking

Fine-Tuning CNN Image Retrieval with No Human Annotation

Word Translation Without Parallel Data

Near-Duplicate Video Retrieval with Deep Metric Learning

Billion-Scale Similarity Search with GPUs

Large-Scale Image Retrieval with Attentive Deep Local Features

Image Copy Detection Based on Convolutional Neural Networks

Bags of Local Convolutional Features for Scalable Instance Search

Practical Black-Box Attacks against Machine Learning

Temporal Matching Kernel with Explicit Feature Maps

YFCC100M

INSTRE

Visual Instance Retrieval with Deep Convolutional Networks

Explaining and Harnessing Adversarial Examples

VCDB: A Large-Scale Database for Partial Copy Detection in Videos

Intriguing properties of neural networks

To Aggregate or Not to aggregate: Selective Match Kernels for Image Search

CoMoFoD — New database for copy-move forgery detection

Event Retrieval in Large Video Collections with Circulant Temporal Encoding

Using extreme value theory for image detection

Aggregating Local Image Descriptors into Compact Codes

TRECVID 2011 - An Overview of the Goals, Tasks, Data,Evaluation Mechanisms, and Metrics

Multiple feature hashing for real-time large scale near-duplicate video retrieval

Exploiting descriptor distances for precise image search

Hello neighbor: Accurate object retrieval with k-reciprocal nearest neighbors

Spatial coding for large scale partial-duplicate web image search

Compact Video Description for Copy Detection with Precise Temporal Alignment

Large-scale image retrieval with compressed Fisher vectors

An Image-Based Approach to Video Copy Detection With Spatio-Temporal Post-Filtering

Logo retrieval with a contrario visual query expansion

Scalable detection of partial near-duplicate videos by visual-temporal consistency

Evaluation of GIST descriptors for web-scale image search

A family of contextual measures of similarity between distributions with application to image retrieval

A survey of the image copy detection

Hamming Embedding and Weak Geometric Consistency for Large Scale Image Search

Lost in quantization: Improving particular object retrieval in large scale image databases

Total Recall: Automatic Query Expansion with a Generative Feature Model for Object Retrieval

Practical elimination of near-duplicates from web video search

UQLIPS: A Real-time Near-duplicate Video Clip Detection System

Object retrieval with large vocabularies and fast spatial matching

Video Histogram: A Novel Video Signature for Efficient Web Video Duplicate Detection

Scalable Recognition with a Vocabulary Tree

Towards effective indexing for very large video sequence database

Image quality assessment: from error visibility to structural similarity

Video Google: a text retrieval approach to object matching in videos

Content-based image copy detection

Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope

Effective and Efficient Global Context Verification for Image Copy Detection

Neural Codes for Image Retrieval

Ieee Transactions on Information Forensics and Security an Evaluation of Popular Copy-move Forgery Detection Approaches

and N

Muscle-vcd-2007: a live benchmark for video copy detection

Inside the team at facebook that dealt with the christchurch shooting

Field of Study

Journal Information

Name