Zero-Shot Cross-Modal Retrieval is the task of finding relevant items across different modalities without having received any training examples. For example, given an image, find a text or vice versa. The main challenge in the task is known as the heterogeneity gap: since items from different modalities have different data types, the similarity between them cannot be measured directly. Therefore, the majority of methods published to date attempt to bridge this gap by learning a latent representation space, where the similarity between items from different modalities can be measured. Source: Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study
image-retrieval-with-multi-modal-query
Zero-Shot Cross-Modal Retrieval
3260 papers • 126 benchmarks • 313 datasets
(Image credit: Papersgraph)
Benchmarks
These leaderboards are used to track progress in image-retrieval-with-multi-modal-query
Trend
Dataset
Best Model
Actions
Flickr30k
Flickr30k
COCO 2014
COCO 2014
Libraries
iUse these libraries to find image-retrieval-with-multi-modal-query models and implementations
3 papers 8,248
Datasets
Subtasks
No subtasks available.
Most implemented papers
Learning Transferable Visual Models From Natural Language Supervision
I. Sutskever, Alec Radford, Jong Wook Kim, Chris Hallacy, A. Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger•Thu Feb 25 2021
It is demonstrated that the simple pre-training task of predicting which caption goes with which image is an efficient and scalable way to learn SOTA image representations from scratch on a dataset of 400 million (image, text) pairs collected from the internet.
39743